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I.  PURPOSE/ INTRODUCTION 


A.  GENERAL  INFORMATION 

The  primary  standards  cited  in  Department  of  Defense  (DoD) 
contracts  for  quality  assurance  programs  are  Military 
Specifications  MIL-Q-9858A  (MIL-Q)  and  MIL-I-45208A  (MIL-I) . 

The  emergence  of  a  global  marketplace  has  led  to  the 
creation  of  quality  control  standards  that  will  apply  to  this 
new  marketplace.  The  International  Standards  Organization 
(ISO)  has  developed  the  ISO  9000  Quality  Management  and 
Quality  Assurance  Standards ( ISO  9000).  These  new 
international  standards  are  applicable  to  countries  in  Europe 
that  will  form  the  European  Community  (EC)  in  1992. 

B.  OBJECTIVE  OF  THE  RESEARCH 

The  purpose  of  this  study  is  to  determine  if  DoD  should 
enact  the  implementation  of  ISO  9000  standards  in  weapon 
systems  contracts.  The  primary  research  question  is:  Should 
ISO  9000  be  implemented  within  DoD  and  the  Defense  Industrial 
Base,  and  when  implemented  how  will  it  affect  DoD  contracting? 
In  addition  to  the  primary  research  question,  the  following 
subsidiary  research  questions  will  also  be  addressed: 

1.  What  is  ISO  9000? 
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2.  What  are  the  similarities  and  differences  between  ISO 
9000  standards  and  Military  Standards  (MIL-Q-9858A  and  MIL- 
1-45208)? 

3.  What  is  the  policy  of  DoD  with  regards  to  ISO  9000? 

4.  Of  companies  within  the  Defense  Industrial  Base  that  have 
completed  ISO  9000  certification,  what  are  the  anticipated 
benefits? 

5.  Should  ISO  9000  be  recommended  for  implementation  within 
DoD  and  the  Defense  Industrial  Base,  and  if  so,  what  action 
must  be  taken  to  accomplish  this  objective? 

C.  SCOPE  AND  LIMITATIONS  OF  THE  STUDY 

This  study  involves  a  detailed  assessment  of  ISO  9000 
standards  as  compared  to  the  current  Military  Standards.  The 
study  involves  gathering  information  from  Government  and 
commercial  organizations  to  determine  the  current  state  of 
knowledge  as  well  as  institutional  forces  that  either  support 
or  reject  the  implementation  of  ISO  9000  in  place  of  Military 
Standards.  The  study  is  not  limited  as  to  size  of  defense 
contractors,  or  to  a  specific  type  of  industry;  suggested 
recommendations  for  implementation  are  applicable  to  the 
entire  Defense  Industrial  Base. 

D .  METHODOLOGY 

The  research  data  were  collected  through  an  extensive 
literature  search  and  by  telephone  interviews. 

The  literature  search,  comprising  professional  journal 
articles  and  current  regulations  and  directives,  was  made 
through  the  Naval  Postgraduate  School  Library,  the  Defense 
Logistics  Studies  Information  Exchange  (DLSIE) ,  attendance  at 
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an  ISO  9000  workshop  sponsored  by  the  University  of  Wisconsin- 
Whitewater,  and  from  several  DoD  offices  responsible  for 
Quality  Assurance.  This  survey  provides  a  background  of 
information  that  defines  and  describes  both  ISO  9000  standards 
and  MIL-STDS  that  apply  to  quality  assurance  systems. 

The  researcher  completed  25  telephone  interviews  with 
managers  and  directors  of  contractor  quality  assurance 
programs.  Appendix  A  provides  a  listing  of  the  companies 
contacted.  The  interviews  were  all  held  on  a  nonattribution 
basis  in  order  to  obtain  candid  responses  and  honest 
evaluations  of  current  and  proposed  implementation  plans  for 
ISO  9000  standards.  The  interviews  were  conducted  by 
telephone  to  allow  the  researcher  to  obtain  and  explore 
information  from  people  who  had  extensive  experience  in 
implementing  ISO  9000  standards. 

E.  ORGANIZATION  OF  THE  STUDY 

Chapter  II  provides  background  information  and 
descriptions  of  ISO  9000  standards  and  Military  Quality 
Assurance  standards. 

Chapter  III  compares  the  underlying  philosophies  of  ISO 
9000  to  Military  Standards  and  draws  detailed  comparisons  of 
the  two  sets  of  standards. 

Chapter  IV  presents  data  and  a  description  of  methods  used 
to  collect  the  information.  This  chapter  also  includes  an 
analysis  and  interpretation  of  the  data. 
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Chapter  V  contains  conclusions  and  recommendations 
regarding  the  implementation  of  ISO  9000  as  well  as  areas  for 
further  research.  While  answering  research  questions,  this 
final  chapter  also  discusses  the  conclusions  based  upon 
research  data,  makes  specific  recommendations  resulting  from 
the  research  effort,  and  suggests  possible  areas  for  further 
research . 
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II.  BACKGROUND 


A.  INTRODUCTION 

The  decline  in  DoD  procurement  dollars  has  led  many 
companies  to  search  for  new  markets  overseas.  As  this 
expansion  into  the  international  market  occurs,  it  is  logical 
that  DoD  contractors  will  argue  for  increased  standardization 
between  current  United  States  standards  and  international 
standards  used  in  contracting. 

One  of  the  primary  areas  of  interest  for  standardization 
lies  in  Quality  Assurance.  The  area  of  quality  control  should 
be  one  in  which  the  buyer  and  seller  agree  on  an  acceptable 
method,  since  standardization  of  quality  assurance  systems 
across  international  boundaries  has  the  potential  of 
significantly  reducing  quality  inspection  costs.  These 
standardization  agreements  create  special  problems  but  hold 
great  potential  when  viewed  from  the  standpoint  of 
international  trade.  In  order  to  achieve  the  desired  quality 
of  products  between  competing  producers,  adherence  to 
international  standards  gives  the  purchaser  an  opportunity  to 
reduce  duplication  of  quality  inspections  without  sacrificing 
the  quality  of  the  end  product.  Additionally,  an 
international  standard  that  is  understood  and  acceptable  to 
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the  buyer  reduces  the  complexity  of  assessing  a  potential 
supplier's  quality  assurance  progrant. 

The  creation  of  the  European  Economic  Community  in  1992 
has  led  to  the  formation  of  international  quality  standards 
(ISO  9000  standards) .  This  standardization  of  quality 
assurance  in  Europe  will  certainly  impact  U.S.  firms  and  the 
DoD  industrial  base.  The  acceptence  of  ISO  9000  standards  for 
use  in  DoD  contracts  has  already  been  proposed  within  DoD. 
The  next  step  in  implementing  ISO  9000  becomes  one  of 
educating  and  familiarizing  both  industry  and  Government 
personnel  in  the  use  of  these  standards. 

The  motivation  to  accept  international  standards  is  based 
on  the  need  for  market  expansion.  As  companies  expand  their 
foreign  sales,  they  will  be  forced  to  adopt  ISO  9000  in  order 
to  remain  competitive  in  overseas  markets.  The  use  of  one 
quality  standard  for  both  DoD  and  international  sales  promises 
reduced  administrative  and  maintenance  costs  relating  to 
quality  assurance  programs. 

The  next  section  of  this  study  will  provide  a  broad 
background  in  both  ISO  and  MIL-STD  Quality  Assurance 
standards . 

B.  ISO  9000  STANDARDS 

The  ISO  9000  standards  comprise  five  (5)  separate 
documents.  A  listing,  accompanied  by  a  brief  description,  is 
as  follows: [Ref  l:p.23] 
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•  ISO  8402.  Quality,  Vocabulary.  This  is  simply  a 
reference  document  that  defines  terms  used  in  ISO  9000 
standards . 

•  ISO  9001.  Quality  Systems.  Model  for  quality  assurance 
in  design/development,  production,  installation  and 
servicing.  This  is  a  specific  model  for  companies  that 
have  all  phases  of  the  manufacturing  process  from  design 
to  final  product,  and  is  the  most  comprehensive  of  the  ISO 
standards . 

•  ISO  9002.  Quality  Systems.  Model  for  quality  assurance 
in  production  and  installation.  This  is  a  specific  model 
for  manufacturing  companies  that  have  all  phases  of 
manufacture  except  design  of  the  product. 

•  ISO  9003.  Quality  Systems.  Model  for  quality  assurance 
in  final  inspection  and  test.  This  gives  specifics  for 
end-item  inspection  procedures. 

•  ISO  9004.  Quality  Management  and  Quality  System  Elements 
Guidelines.  This  is  another  reference  document  that 
explains  the  philosophy  and  underlying  purpose  of  ISO 
9000. 


The  ISO  9000  standards  are  based  on  an  approach  to  quality 
assurance  that  models  itself  along  Total  Quality  Management 
(TQM)  principles.  The  aim  of  the  standards  was  summarized  by 
Trevor  Davis  from  the  quality  management  group  at  Coopers  & 
Lybrand  Deloitte,  as  follows: [Ref  l:p.25] 


•  to  increase  customer  confidence  in  the  company,  by 
providing  a  common  framework  across  Europe  (and  the 
world)  ; 

•  to  move  from  a  system  of  inspection  to  one  of  quality 
management ; 

•  to  remove  the  need  for  multiple  assessments  of  suppliers; 

•  to  gain  management  commitment ; to  link  quality  to  cost 
effectiveness;  and 

•  to  give  customers  what  they  have  asked  for. 
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The  above  summary  highlights  some  of  the  main  themes  that 
run  through  the  ISO  9000  standards.  Many  of  the  principles 
listed  clearly  reflect  the  TQM  approach. 

Each  of  the  ISO  9000  documents  covers  different  phases  of 
the  quality  assurance  area,  and  each  document  is  briefly 
described  below. 

1.  ISO  9004,  Quality  Management :  Specification  for 
Design /Development,  Production,  Installation  and 
Servicing. 

The  document  that  gives  broad  guidelines  and  amplifies 

the  other  four  ISO  documents  is  ISO  9004.  The  ISO  9000 

standards  were  written  with  two  applications  in  mind.  The 
first  was  the  creation  of  guidance  for  companies  to  use  in 
developing  their  quality  assurance  organizations;  the  second, 
a  program  that  would  satisfy  the  contractual  quality 
requirements  dictated  by  customers . [Ref  2:p.2] 

The  ISO  9004  document  provides  guidance  in  creating  an 
internal  quality  assurance  organization.  The  document  begins 
with  a  list  of  definitions  which  incorporates  ISO  8402 

definitions  into  the  ISO  9000  standards.  ISO  9004  also 
outlines  concepts  regarding  principles  of  Quality  Assurance 
and  the  proper  selection  of  ISO  9001,  9002  or  9003  for 

specific  applications.  ISO  9004  concludes  by  bridging  the 
internal/external  applications  of  the  ISO  9000  standards.  It 
clearly  calls  for  the  use  of  customized  versions  of  the  9001, 
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9002,  and  9003  documents.  The  ISO  9000  documents  state  that 
it  is  not  the  intention  of  ISO  to  force  suppliers  into  the 
acceptance  of  one  standardized  quality  assurance  organization. 
Rather,  the  standards  are  meant  to  assure  both  the  supplier 
and  customer  that  a  product /service  is  being  provided  that 
will  satisfy  contractual  requirements  for  quality. 

2.  ISO  9001,  Quality  Systems:  Specification  for 

Design/Development ,  Production,  installation  and 

Servicing. 

This  portion  of  ISO  9000  is  intended  "for  use  when 
conformance  to  specified  requirements  is  to  be  assured  by  the 
supplier  during  several  stages  which  may  include  design/ 
development,  production,  installation  and  servicing ." [Ref 
4:p.l]  Paragraph  4  of  this  portion  of  ISO  9000  gives  an 
outline  of  a  quality  assurance  program  that  takes  the 
following  form: 

•  4.1  Management  responsibility 

•  4.2  Quality  system 

•  4.3  Contract  review 

•  4.4  Design  control 

•  4.5  Document  control 

•  4.6  Purchasing 

•  4.7  Purchaser  supplied  product 

•  4.8  Product  identification  and  traceability 

•  4.9  Process  control 
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•  4.10  Inspection  and  testing 

•  4.11  Inspection,  measuring  and  test  equipment 

•  4.12  Inspection  and  test  status 

•  4.13  Control  of  nonconforming  product 

•  4.14  Corrective  action 

•  4.15  Handling,  storage,  packaging  and  delivery 

•  4.16  Quality  records 

•  4.17  Internal  quality  audits 

•  4.18  Training 

•  4.19  Servicing 

•  4.20  Statistical  techniques 

The  ISO  9001  document  is  written  in  simple,  easy-to- 
understand  language,  and  many  of  the  paragraphs  listed  above 
are  subdivided  into  parts  composed  of  one  sentence.  As  an 
example,  paragraph  4.15.2,  "Handling:” 

The  supplier  shall  provide  methods  and  means  of  handling 

that  prevent  damage  or  deterioration. [Ref  3:p.6] 

Numerous  other  examples  can  be  seen  in  Appendix  B 
which  provides  a  comparison  of  ISO  9001  to  MIL-Q-9858A.  It 
should  be  noted  that  Appendix  B  lists  ISO  9001  paragraphs  in 
the  right-hand  column  of  the  appendix,  and  the  entire  appendix 
is  structured  using  the  format  of  ISO  9001. 

ISO  9001  is  a  model  for  a  quality  assurance 
organization  that  is  engaged  in  design,  development  and 
production  of  a  complex  product  requiring  conformance  to 
buyer-directed  specifications.  It  does  not  give  a  micro- 
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management  approach  to  the  quality  program  being  used  by  the 
supplier.  Suppliers  are  free  to  implement  the  model  in  a  way 
that  best  suits  their  production  process.  Yet  the  model  does 
cover  all  the  areas  of  a  sound  quality  assurance  program. 

ISO  9001  is  meant  to  be  a  model,  designed  for  systems 
procurements  that  include  the  design  of  the  end  product  by  the 
manufacturer.  It  covers  all  aspects  of  producing  a  complete 
weapon  system  as  specified  by  a  buyer.  The  supplier  is 
expected  to  conform  to  buyer  specifications  in  several  phases 
of  the  procurement.  The  ISO  9002  standard  takes  the  systems 
approach  and  reduces  it  to  an  established  product  line. 

3.  ISO  9002,  Quality  Systems:  Specification  for 

Production  and  Installation. 

The  functions  of  design/development  and  servicing  have 
been  removed  from  ISO  9001  in  order  to  produce  the  ISO  9002 
document.  This  portion  of  ISO  9000  is  intended  "for  use  when 
conformance  to  specified  requirements  is  to  be  assured  by  the 
supplier  during  production  and  installation .” [Ref  4:1]  The 
outline  of  the  model  in  this  document  is  composed  along  the 
same  lines  as  shown  in  paragraph  B  above,  with  the  removal  of 
subparagraphs  Design  Control  and  Servicing.  The  ISO  9002 
document  is  very  similar  in  detail  to  ISO  9001,  except  it 
assumes  a  mature  product  with  a  design  not  subject  to  much 
change . 
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The  ISO  9003  document  takes  the  ISO  9002  and  reduces 


it  even  further  to  where  it  becomes  appropriate  for  use  in 
established  production  lines  that  are  built  to  producer 
specifications . 

4.  ISO  9003/  Quality  Systems:  Specification  for  Final 
Inspection  and  Test. 

This  portion  of  ISO  9000  is  intended  "for  use  when 
conformance  to  specified  requirements  is  to  be  assured  by  the 
supplier  solely  at  final  inspection  and  test. "[Ref  5:p.l] 
Paragraph  4  of  ISO  9003  outlines  a  quality  assurance  program 


as 

follows : 

e 

4.1 

Management  responsibility 

e 

4.2 

Quality  system 

e 

4.3 

Document  control 

e 

4.4 

Product  identification 

e 

4.5 

Inspection  and  testing 

e 

4.6 

Inspection,  measuring  and  test  equipment 

# 

4.7 

Inspection  and  test  status 

e 

4.8 

Control  of  nonconforming  product 

• 

4.9 

Handling,  storage,  packaging  and  delivery 

• 

4.10 

Quality  records 

e 

4.11 

Training 

e 

4.12 

Statistical  techniques 

12 


ISO  9003  obviously  applies  to  mass  production  of 
supplier-specified  products.  In  addition  to  the  deletion  of 
subparagraphs  in  paragraph  4  that  relate  to  process 
inspection,  the  remainder  of  ISO  9003  has  been  modified  to 
reflect  a  less  intrusive  quality  assurance  program  regarding 
purchasing  and  process  control.  Using  ISO  9003  puts  very 
little  burden  on  the  supplier  other  than  final  inspection  and 
test  requirements. 

C.  DOD  MILITARY  QUALITY  STANDARDS 

The  primary  standards  cited  in  Department  of  Defense  (DoD) 
contracts  concerning  quality  assurance  programs  are  Military 
Specifications  MIL-Q-9858A  (MIL-Q)  and  MIL-I-45208A  (MIL-I) . 
A  brief  overview  of  the  guiding  philosophy  for  each  of  these 
specifications  is  presented  below.  This  overview  is  intended 
to  provide  background  for  comparison  of  these  DoD 
specifications  to  the  ISO  9000  standards;  the  comparison  will 
be  done  in  the  next  section  of  this  study. 

1.  MIL-Q-9858A:  Quality  Program  Requirements. 

This  specification  is  structured  from  the  standpoint 
of  the  Government.  It  directs  suppliers  on  what  they  shall 
consider  when  conforming  to  this  specification,  and  it 
requires  approval  of  the  company's  quality  program  by  the 
Government.  The  following  two  paragraphs  provide  a 
perspective  for  this  specification: 


13 


1.2  Contractual  Intent.  This  specification  requires  the 
establishment  of  a  quality  program  by  the  contractor  to 
assure  compliance  with  the  requirements  of  the  contract. 
The  program  and  procedures  used  to  implement  this 
specification  shall  be  developed  by  the  contractor.  The 
quality  program,  including  procedures,  processes  and 
product  shall  be  documented  and  shall  be  subject  to  review 
by  the  Government  Representative.  The  quality  program  is 
subject  to  the  disapproval  of  the  Government 
Representative  whenever  the  contractor's  procedures  do  not 
accomplish  their  objectives.  The  Government,  at  its 
option,  may  furnish  written  notice  of  the  acceptability  of 
the  contractor's  quality  program. 

1.3  Summary.  An  effective  and  economical  quality  program, 
planned  and  developed  in  consonance  with  the  contractor's 
other  administrative  and  technical  programs,  is  required 
by  this  specification.  Design  of  the  program  shall  be 
based  upon  consideration  of  the  technical  and 
manufacturing  aspects  of  production  and  related 
engineering  design  and  materials.  The  program  shall 
assure  adequate  quality  throughout  all  areas  of  contract 
performance;  for  example,  design,  development, 
fabrication,  processing,  assembly,  inspection,  test, 
maintenance,  packaging,  shipping,  storage  and  site 
installation . 


These  two  examples  illustrate  the  somewhat  negative 
writing  style  and  all-encompassing  nature  of  MIL-Q-9858A 
specification.  There  are  "laundry  lists"  throughout  the 
specification  written  to  cover  all  possible  eventualities  that 
might  affect  the  quality  program  of  a  company  engaged  in 
complex  production  processes.  MIL-Q-9858A  (which  is 
reproduced  on  the  left-hand  side  of  Appendix  B)  offers  a 
legalistic  framework  in  much  of  its  phrasing  and  words.  A 
careful  analysis  of  MII--Q-9858A  leads  one  to  conclude  that  the 
drafters  of  this  specification  were  seeking  to  close  as  many 
legal  challenges  to  their  idea  of  a  quality  program  as 
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possible.  The  goal  of  using  the  existing  supplier  quality 
assurance  program  to  fulfill  quality  requirements  of  the 
Governmens  becomes  lost  in  all  the  Government  oversight  and 
inspections,  required  to  conform  to  this  specification.  The 
end  result  is  a  requirement  on  the  part  of  the  Government  to 
be  constantly  inspecting  the  quality  assurance  program  to 
ensure  compliance  with  all  the  various  parts  of  this 
specif ication .  Thus  MIL-Q-9858A  becomes  an  attempt  to 

"inspect  quality  into  the  product"  rather  than  build  it  in, 
using  good  quality  processes  that  stress  improving  the 
existing  production  system. 

This  specification  becomes  even  more  encumbering  and 
intrusive  when  references  within  MIL-Q  are  reviewed.  A 
partial  list  of  these  specifications,  along  with  brief  titles, 
is  given  below: 

•  MIL-I-45208A  --  Inspection  system  requirements 

•  MIL-C-45662  --  Calibration  system  requirements 

•  MIL-STD-105  --  Sampling  procedures  and  tables  for 

inspection  by  attributes 

•  MIL-STD-109  --  Quality  assurance  terms  and  definitions 

This  provides  much  more  than  broad  guidance  in  how  a 
supplier  will  establish  his  quality  assurance  organization. 
While  the  specification  clearly  says  the  contractor  is 
responsible  for  this  system,  DoD  is  stipulating  rules  of 
operation  throughout  MIL-Q-9858A  that  limit  a  contractor's 
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options.  The  scope  of  this  specification  is  revealed  in  Note 
8.1: 

8.1  Intended  Use.  This  specification  will  a£ply  to 
complex  supplies,  components,  equipments,  and  systems  for 
which  the  requirements  of  MIL-I-45208  are  inadequate  to 
provide  needed  quality  assurance.  In  such  cases,  total 
conformance  to  contract  requirements  cannot  be  obtained 
effectively  and  economically  solely  by  controlling 
inspection  and  testing.  Therefore,  it  is  essential  to 
control  work  operations  and  manufacturing  processes  as 
well  as  inspections  and  tests. 

This  quote  clearly  shows  that  the  intent  of  MIL-Q-9858A  is  to 
obtain  total  control  over  the  process  and  operation  of  the 
supplier.  There  is  an  implication  that  if  DoD  inspects  both 
the  end  product  and  the  process  a  sufficient  number  of  times, 
the  quality  of  the  product  will  eventually  meet  its 
expectations . 

In  contrast  to  the  idea  of  total  control,  DoD  has 
promulgated  MIL-I-45208A  for  those  contracts  that  do  not  need 
to  be  as  closely  monitored  because  of  the  nature  or  complexity 
of  the  product . 

2.  MIL- I -4 520 8A:  inspection  System  Requirements. 

This  specification  contains  fewer  requirements  than 
MIL-Q,  but  it  is  still  based  on  inspection  and  detailed 
accounting  of  quality  by  the  supplier.  There  is  an  attempt  to 
cover  as  many  contingencies  as  possible  and  to  provide  not 
only  guidance,  but  decisions  concerning  possible  scenarios 
that  might  arise  during  the  performance  of  a  contract.  These 
decisions  take  the  form  of  negative  wording  regarding  any 
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deviations  or  foreseeable  problems.  An  example  is  found  in 
paragraph  2.2: 

2.2  Amendments  and  Revisions.  Whenever  this  specification 
is  amended  or  revised  subsequent  to  its  contractually 
effective  date,  the  contractor  may  follow  or  authorize  his 
subcontractors  to  follow  the  amended  or  revised  document 
provided  no  increase  in  price  or  fee  is  required.  The 
contractor  shall  not  be  required  to  follow  the  amended  or 
revised  document  except  as  a  change  in  contract.  If  the 
contractor  elects  to  follow  the  amended  or  revised 
document,  he  shall  notify  the  Contracting  Officer  in 
writing  of  this  election.  When  the  contractor  elects  to 
follow  the  provisions  of  an  amendment  or  revision,  he  must 
follow  them  in  full. 

This  paragraph  shows  how  the  drafters  of  MIL-I-45208 
tried  to  incorporate  all  possible  scenarios  of  a  change  to  a 
product  within  the  specification.  The  specification  first 
states  that  a  contractor  may  elect  to  follow  a  revision, 
provided  there  is  no  increase  in  price.  Then  it  is 
hypothesized  that  a  contractor  might  elect  to  follow  only  part 
of  the  revision  and  so  that  becomes  forbidden.  The  hypothesis 
is  taken  one  step  further  in  assuming  a  contractor  will  fail 
to  tell  the  Government  what  course  of  action  has  been 
implemented  regarding  amendments  or  revisions.  To  preclude 
this  possible  oversight  on  the  part  of  a  contractor,  MIL-I- 
45208  requires  that  the  Contracting  Officer  be  notified  in  the 
event  an  amendment  or  revision  is  implemented.  A  careful 
reading  of  this  paragraph  shows  how  the  original  intent  of  the 
specification,  which  meant  simply  to  state  that  a  contractor 
does  not  have  to  follow  revisions  until  they  are  formally 
added  to  a  contract,  has  become  an  exercise  in  semantics. 
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This  type  of  phrasing  and  writing  pervades  both  MIL-Q-9858A 
and  MIL-I-452 08 . 

D.  LITERATURE  REVIEW 

In  order  for  the  researcher  to  understand  the 
attraction  of  using  international  standards,  and  the  movement 
within  DoD  to  accept  them,  a  survey  of  current  literature 
regarding  international  quality  standardization  was  conducted. 

1.  Change  in  attitudes  during  1980s. 

The  early  1980s  saw  the  realization  that  the 
predominate  position  of  the  United  States  as  arms  supplier  to 
all  its  allies  --  in  particular,  its  European  allies  --  was 
being  seriously  challenged.  The  political  climate  dictated 
that  jobs  and  technology  development  in  Europe  be  protected  by 
the  European  governments.  This  political  drive  to  give  equal 
status  to  the  European  arms  industrial  base  led  to  serious 
concerns  that  soon  the  North  Atlantic  Treaty  Organization 
(NATO)  would  be  equipped  with  incompatible  weapons  across 
national  boundaries.  These  concerns  gave  rise  to  the  concept 
of  Rationalization,  Standardization  and  Interoperability 
(RSI) .  The  political  climate  of  both  Europe  and  the  U.S.  was 
intent  on  achieving  maximum  military  effectiveness  for  the 
money  expended.  One  way  of  achieving  this  effectiveness  was 
to  create  joint  efforts  that  reduced  the  duplication  of 
research  and  development  funds  while  increasing  the  potential 
market  and  numbers  of  weapons  eventually  deployed.  At  the 
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same  time,  the  early  1980s  saw  the  U.S.  as  the  dominant 
producer  of  complex  weapon  systems  for  the  free  world.  Along 
with  this  dominance  came  a  certain  amount  of  parochial 
interest  on  the  part  of  American  companies  and  DoD  components 
dealing  in  the  international  arena.  The  U.S.  Government  and 
industrial  base  were  viewed  by  our  allies  as  using  defense  to 
protect  and  justify  purely  economic  decisions. 

[Ref  6:p.7] 

This  attitude,  that  the  U.S.  would  establish  contracts 
with  NATO  allies  to  maximize  resource  use  and  to  ensure  NATO 
compatibility  in  weapon  systems,  has  been  replaced.  The  new 
views  on  international  contracting  have  to  do  with  the 
realization  that  the  United  States  is  •.  dominant  in  many 
areas  and  thus  must  compete  fci  its  foreign  market  share. 
This  new  attitude  is  summarized  in  the  following  citation:  [Ref 
7 :p . 6] 


This  year's  NCM  Week  slogan  (Contracting  for  a  Competitive 
America)  is  not  prophetic;  it  is  descriptive  of  current 
real  world  events.  The  general  downsizing  of  defense  and 
high  technology  markets  and  the  increasing  competition 
from  essentially  equivalent  foreign  competitors  are 
demonstrable  indicators  that  the  economic  future  offers 
rewards  to  those  who  succeed  in  international  competition 
and  penalties  (in  the  form  of  decreased  market  share, 
fewer  jobs,  and  less  profit)  to  those  who  do  not. 

International  arms  deals  are  now  viewed  as  a 
competitive  field.  Industry  and  DoD  now  see  foreign 
competitors  as  more  of  an  equal  and  thus  the  U.S.  can  no 
longer  dictate  the  terms  of  international  arms  sales. 
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2 .  Policy  statements  regarding  quality  assurance  in 

international  contracts. 

The  official  DoD  policy  toward  adoption  of  ISO  9000 
can  be  seen  in  several  documents.  First,  at  a  broad  policy 
level,  there  is  DoD  Instruction  5000.2  Part  6  Section  Q  (c) 
Participation  in  Standards  Development  Activities,  which 
states : 

DoD  Components  will  participate  in  standards  development 
activities  of  non-Government  standards  bodies,  both 
domestic  and  international,  coordinating  on  such  activity 
with  other  Federal  Agencies. 

This  policy,  in  the  case  of  ISO  9000  standards,  has 
been  implemented  through  its  inclusion  in  the  Department  of 
Defense  Index  of  Specifications  and  Standards  (DODISS)  list  of 
acceptable  documents  for  use  in  contracts. 

The  DoD  5000.2  is  backed  up  with  an  Assistant 
Secretary  of  Defense  memorandum  dated  7  August  1989  that 
states:  [Ref  8:p.l] 

I  have  been  briefed  on  the  series  of  quality  assurance 
standards  recently  published  by  the  International 
Organization  for  Standardization  (ISO).  After  reviewing 
the  benefits  of  adopting  these  ISO  quality  assurance 
standards,  their  wide  acceptance  around  the  world,  and 
their  implications  in  international  trade,  I  have  decided 
to  follow  an  approach  similar  to  that  taken  by  NATO.  I 
believe  that  this  approach  will  provide  maximum  benefit  to 
the  Department  of  Defense  (DoD)  as  well  as  American 
industry . 

More  specifically,  I  want  to  adopt  ISO  standards,  9001, 
9002,  9003  in  their  entirety  and  develop  supplemental 
military  standards  that  will  incorporate  the  appropriate 
ISO  standards  by  reference  and  provide  the  requirements 
for  a  contractor  quality  program  which  may  be  lacking  in 
the  ISO  standards. 


20 


The  Army  is  the  Service  branch  with  cognizance  over 
Quality  Assurance  for  DoD,  and  thus  has  the  lead  in  policy 
generation  and  modification  for  quality  assurance. 

In  a  brief  given  by  the  Assistant  Director,  Office, 
Secretary  of  Defense  International  Quality  Assurance,  the 
following  bullets  are  .  listed  on  a  viewgraph  labeled  "U.S. 
National  Position  on  ISO  QA  Standards . " [Ref  9] 

•  US  has  adopted  ISO  QA  Standards:  ANSI/ASQA  Q-90  Series 

•  US  consensus  is  ISO  Standards  are  insufficient 

•  ISO  Standards  will  be  revised  or  reconfirmed  in  1992 

•  Task  force  to  review  Q-90  Series  Standards  established 

•  Task  force  authorized  under  ANSI  ASC  A-l  committee 

•  Task  force  to  identify  improvements  in  ISO  standards 

•  These  improvements  will  be  provided  to  US  TAG  [Technical 
Advisory  Group]  and  to  ISO  TC176  [Technical  Committee] . 

•  Task  force  has  met  several  times 

•  DoD  and  defense  industry  are  represented 

The  U.S.  Department  of  Commerce  is  also  heavily 
involved  in  the  promulgation  of  information  regarding  ISO 
Standards  and  in  assessing  the  impact  they  will  have  when  the 
European  Community  is  formally  created  in  1992.  The  abstract 
shown  below  is  from  an  article  published  by  the  Department  of 
Commerce,  National  Institute  of  Standards  and  Technology, 
Standards  Code  and  Information  Program,  Office  of  Standards 
Service:  [Ref  10:p.  ii] 
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This  report  provides  information  on  the  development, 
content  and  application  of  the  ISO  9000  standards  to 
readers  who  are  unfamiliar  with  these  aspects  of  the 
standards.  It  attempts  to  answer  some  of  the  most 
commonly  asked  questions  on  quality;  quality  systems;  the 
content,  application  and  revision  of  the  ISO  9000 
standards;  quality  system  approval/registration;  European 
Community  requirements  for  quality  system 
approval/registration;  and  sources  for  additional  help. 

The  literature  cited  above  is  meant  to  give  a  general 

sense  of  what  the  U.S.  Government  and  in  particular  the  DoD 

position  is  with  regards  to  ISO  9000.  It  is  certainly  not  an 

exhaustive  survey  of  all  the  policy  issues  and  actions  taken 

or  proposed,  but  it  does  represent  the  largest  force  within 

this  area  of  Quality  Assurance. 

3.  Concerns  regarding  the  use  of  ISO  9000  and  the  manner 
in  which  ISO  9000  Standards  are  changed. 

During  the  literature  review,  documents  that  outline 
the  concerns  and  perceived  deficiencies  in  ISO  9000  standards 
were  located.  These  documents  are  best  summarized  by  a 
concise  overview  used  during  a  briefing  entitled  "ISO  9000 
Quality  Assurance  standards"  given  during  a  training  session 
for  Quality  Assurance  Representatives  (QARs)  in  the  Defense 
Contract  Management  District  (DCMD)  Northeast.  The  overview 
was  titled  "ISO  QA  Standards  Concerns"  and  listed  the 
following  bullets: [Ref  ll:p.l2] 


•  Contractor  certification 

•  Requirements  not  adequately  covered  by  ISO/ANSI  Standards 

•  DoD  guidance  document  [to  be  issued  in  near  future] 
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•  Training  [QAR  training  in  philosophy  and  application  of 
ISO  Standards] 

The  issues  regarding  deficiencies  in  ISO  9000 
Standards  are  the  purview  of  the  International  Standards 
Organization  Technical  Committee  176  (TC176) .  TC176  is 

responsible  for  overseeing  the  modifications  and  additions 
made  to  ISO  9000  Standards.  The  four  Strategic  Goals  of  the 
TC176  were  recently  defined  by  using  "test"  statements.  The 
goals  and  associated  "tests"  are  shown  below. [Ref  12:p.33] 
Goal:  Universal  Acceptance 

Tests  for  the  goal  of  universal  acceptance: 

•  The  standards  are  widely  adopted  and  used,  worldwide. 

•  There  are  few  complaints  from  users  in  proportion  to  the 
volume  of  use. 

•  Few  sector-specific  supplementary  or  derivative  standards 
are  being  used,  or  developed. 

Goal:  Current  Compatibility 

Tests  for  current  compatibility: 

•  "  ^rt  Number"  supplements  to  existing  standards  do  not 
change  or  conflict  with  requirements  in  the  existing 
parent  document . 

•  The  numbering  and  clause  structure  of  a  supplement 
facilitate  combined  use  of  the  parent  document  and  the 
supplement . 

•  Supplements  are  not  stand-alone  documents,  but  a?.*e  to  be 
used  with  their  parent  document. 


Goal:  Forward  Compatibility 


Tests  for  forward  compatibility: 


•  Revisions  affecting  requirements  in  existing  standards  are 
few  in  number  and  minor  or  narrow  in  scope. 

•  Revisions  are  accepted  for  existing  as  well  as  new 
contracts . 


Goal:  Forward  Flexibility 
Tests  for  forward  flexibility: 


•  Supplements  are  few  in  number,  but  can  be  combined  as 
needed  to  meet  the  needs  of  virtually  any 
industry/economic  sector  or  generic  category  of  products. 

•  Supplement  or  addendum  architecture  allows  new  features  or 
requirements  to  be  consolidated  into  the  parent  document 
at  a  substantial  revision,  if  the  supplement's  provisions 
are  found  to  be  used  (almost)  universally. 


The  two  documents  cited  above  illustrate  the  concern 
many  ISO  members  have  regarding  modifications  or  changes  to 
ISO  9000.  The  standards  must  be  applicable  across  national 
boundaries  and  they  must  cover  all  types  of  industry.  ISO 
developed  a  method  for  dealing  with  modifications  of  ISO  9000 
in  order  to  accommodate  the  resolution  of  some  of  these 
concerns.  In  the  case  of  DoD,  the  stated  policy  is  to  issue 
and  impose  additional  requirements,  with  regard  to  the 
contractor's  quality  system  as  needed,  to  ensure  a  contractor 
has  an  acceptable  quality  system  in  place. 
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B.  SUMMARY 


The  structure  and  philosophy  of  ISO  9000  as  explained 
above  is  designed  to  facilitate  continuous  improvement  in 
quality.  Thus  it  is  a  step  closer  to  a  Total  Quality 
Management  system  as  well.  On  the  other  hand,  the  existing 
Military  Standards  tend  to  inspect  quality  into  the  product, 
as  opposed  to  concentration  on  improvement  of  the  process. 

The  literature  search  revealed  a  concerted  policy  within 
DoD  to  encourage  and  educate  both  contractors  and  DoD 
personnel  in  the  application  of  ISO  9000.  At  the  same  time, 
DoD  is  reserving  the  right  to  modify  ISO  9000  programs  by 
imposing  existing  or  new  quality  standards  as  appropriate  for 
a  given  contract . 

The  following  chapter  compares  MIL-Q-9858A  to  ISO  9001  and 
MIL-I -452  08  to  ISO  9002.  This  comparison  highlights 
differences  between  the  two  sets  of  standards,  in  order  to 
gain  a  better  understanding  of  the  changes  that  will  result 
from  imposing  ISO  9000  in  DoD  contracts. 
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III.  COMPARISON 


A.  INTRODUCTION 

The  structure  of  the  ISO  9000  series  of  standards  makes  it 
easy  to  compare  ISO  9001  to  MIL-Q-9858A.  But  ISO  9002  does 
not  compare  well  to  MIL-I-45208,  while  ISO  9003  can  be  thought 
of  as  equivalent  to  inspection  by  the  contractor  of  his  own 
product  or,  more  broadly,  as  commercial  end-item  inspection. 
This  researcher  prepared  a  comparison  of  ISO  9001  to  MIL-Q- 
9858A  and  a  comparison  of  ISO  9002  to  MIL-I-45208.  These 
comparisons  are  presented  in  Appendices  B  and  C  of  this 
thesis . 

Appendix  B  compares  individual  paragraphs  of  ISO  9001  to 
corresponding  paragraphs  of  MIL-Q-9858A,  while  Appendix  C 
compares  ISO  9002  to  MIL-I-45208  on  a  paragraph-by-paragraph 
basis.  Thus,  a  detailed  comparison  of  the  standards  in 
regards  to  a  particular  subject  matter  is  available  by  careful 
scrutiny  of  Appendices  B  and  C.  After  completion  of  these 
comparisons,  the  researcher  discovered  a  similar  collation 
completed  by  the  Office  of  the  Assistant  Secretary  of  Defense 
(Production  and  Logistics).  In  reviewing  the  researcher's 
comparison  as  opposed  to  that  of  the  OSD(P&L),  it  is  evident 
that  the  two  documents  agree  only  in  principle  as  to  possible 
alignments  of  paragraphs  between  ISO  9001  and  MIL-Q-9858A. 
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This  wide  disparity  illustrates  the  subjective  nature  of  both 
documents.  It  is  possible  to  interpret  a  given  paragraph  in 
ISO  or  MIL-STD  in  several  different  ways.  The  OSD(P&L) 
comparison  is  discussed  in  detail  below,  and  Appendix  D  of 
this  thesis  is  a  matrix  of  the  OSD(P&L)  alignment  of 
paragraphs  between  ISO  9001  and  MIL-Q-9858A. 

It  should  be  noted  that  Appendices  B  and  C  adopted  the  ISO 
9000  organization  with  the  right-hand  side  of  the  table 
reflecting  the  ISO  9000  structure.  MIL-Q-9858A  (in  Appendix 
B)  and  MIL-I-45208  (in  Appendix  C)  are  then  applied  to  the  ISO 
Standards  on  a  best-of-fit  type  comparison  on  the  left  side  of 
the  appendix. 

B .  GENERAL  OBSERVATIONS 

After  reviewing  the  detailed  comparison  of  ISO  to  MIL 
STD,  three  general  observations  were  made  by  this  researcher: 

1.  The  ISO  standards  use  the  phrase  "as  appropriate" 
while  the  U.S.  specifications  attempt  to  list  all  possible 
situations  to  which  they  might  wish  to  apply  a  given 
paragraph . 

2.  The  ISO  standards  use  a  clear  outline  format, 
which  separates  the  main  subjects  into  short  easy-to-read 
statements,  while  U.S.  specifications  employ  long  narrative 
paragraphs . 

3.  The  ISO  standards  take  an  overall  approach  and 
provide  broad  management  structure  to  the  programs  they  are 
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seeking  to  encourage.  U.S.  specifications  appear  to  direct 
contractors  to  take  specific  action  in  areas  that  have  caused 
problems  in  the  past. 

A  second  view  of  the  general  differences  between  ISO  9000 
and  Military  Standards  was  given  during  a  panel  discussion  at 
the  23rd  Annual  Industry /Government  Quality  Liaison  Meeting 
held  at  Darr'  rs,  Massachusetts.  The  following  concise 
comparison  of  ISO  9000  to  MIL-Q  strengths  was  presented  as  a 
panel  discussion  subject. [Ref  13 : p . 6 ] 

ISO  9000  and  MIL-0-9858A  are  generally  equivalent 
ISO  9000  is  stronger  in: 

1.  Design  control 

2.  Management  commitment 

3.  Personnel  training/qualification 

4.  Internal  quality  audits 

MIL-O-9858  is  stronger  in: 

1.  Calibration  and  measurement 

2 .  Control  of  nonconforming  supplies 

3 .  Government  review  of  QA  program 

4.  Cost  of  quality 

Both  are  weak  in  continuous  process  improvement. 

C.  RESEARCHER'S  COMPARISON  OF  ISO  9001  TO  MIL-Q-9858A 

ISO  9001  compares  favorably  with  MIL-Q-9858A.  This 
researcher  does  not  detect  any  significant  differences  between 
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the  two.  Both  documents  cover  the  same  areas  of  process  and 
design  control. 

ISO  9001  does  not  currently  contain  the  "layering  effect" 
that  is  inherent  in  MIL-Q-9858A.  This  layering  effect  is 
where  MIL-Q-9858A  refers  to  other  specifications  or  handbooks 
that  amplify  and  add  more  detail  to  MIL-Q-9858A.  There  is 
resistance  to  additional  amplification  within  the  ISO 
organization  and  membership.  As  discussed  above,  the  ISO 
TC176  has  established  a  set  of  strategic  goals  that  are 
designed  to  avoid  the  micro-management  of  MIL-Q  implementation 
practices.  One  of  the  goals  of  ISO  is  to  remain  at  a  basic 
level  that  can  to  many  different  industries  and 
situations . 

D.  RESEARCHER'S  COMPARISON  OF  ISO  9002  TO  MIL-I-45208 

ISO  9002  is  far  more  comprehensive  and  provides  more 
direction  than  does  MIL-I-45208  and  is  not  a  close  replacement 
for  it.  ISO  9002  actually  exceeds  the  MIL-I-45208 
requirements  in  all  areas,  and  is  virtually  identical  to  ISO 
9001  (with  the  removal  of  the  paragraphs  in  ISO  9001 
concerning  design  quality  control). 

All  other  provisions  regarding  process  control  and 
documentation  remain  the  same  between  ISO  9001  and  ISO  9002. 
The  use  of  ISO  9002  in  place  of  MIL-I-45208  has  the  potential 
to  increase  expenditure  of  resources  by  contractors  to  comply 


with  it,  over  and  above  the  costs  of  complying  with  MIL-I- 
45208. 

B.  RESEARCHER'S  ASSESSMENT  OF  SECTIONS  IN  MIL-Q-9858  AND  MIL- 

1-45208  WITH  NO  CORRESPONDING  ISO  SECTIONS 

A  review  of  the  text  for  those  paragraphs  that  have  no 
corresponding  ISO  wording,  reveals  that  these  paragraphs  do 
not  deal  with  defining  a  quality  program.  Rather,  they  are 
directives  designed  to  provide  the  Government  additional  legal 
rights  to  conduct  a  detailed  or  more  intrusive  oversight  of 
subcontractors.  They  also  ensure  legal  right  to  the  use  of 
contractor  equipment  in  conducting  in-process  inspections. 
These  paragraphs  are  not  deemed  by  this  researcher  to  be 
applicable  to  the  overall  effectiveness  of  a  quality  program. 

1.  MIL-Q-9858A. 

The  paragraphs  of  MIL-Q-9858A  that  do  not  have 
corresponding  ISO  9001  paragraphs  are  listed,  along  with  their 
titles,  below: 

•  3.6  Cost  Related  to  Quality 

•  4.4  Use  of  Contractor's  Inspection  Equipment 

•  4.5  Advanced  Metrology  Requirements 

•  7.7  Government  Inspection  at  Subcontractor  Facilities 

The  complete  text  of  these  paragraphs  can  be  found  at 
the  end  of  Appendix  B. 
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2.  MIL-I-45208. 


The  paragraphs  of  MIL-I-45208  that  do  not  have 
corresponding  ISO  9002  paragraphs  are  listed,  along  with  their 
titles,  below: 

•  3.8  Qualified  Products 

•  3.11  Government  Inspection  at  Subcontractor  or 

Vendor  Facilities 

•  3.11.1  Government  Inspection  Requirements 

•  3.13  Government  Evaluation 

The  complete  text  of  these  paragraphs  can  be  found  at 
the  end  of  Appendix  C. 

P.  OSD(P&L)  COMPARISON  OF  ISO  9001  TO  MIL-Q-9858A 

The  OSD ( P&L )  conducted  a  comparison,  resulting  in  38 
comments  on  the  differences  between  ISO  9001  and  MIL-Q-9858A. 
The  comments  have  been  broken  down  by  this  researcher  into 
broad  areas  of  the  standards  to  which  they  apply,  and  are 
presented  below.  This  OSD (P&L)  comparison  is  meant  only  as  a 
reference  document  and  not  as  a  policy  statement. 

1 .  Scope . 

This  is  paragraph  1.0  in  ISO  9001  and  paragraph  1  in 
MIL-Q.  The  paragraphs  are  intended  in  both  standards  to  state 
the  applicability  of  the  standard.  OSD(P&L)  comments  are: 
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•  Mil-Q-9858  states  that  the  contractor  is  responsible  for 
compliance  with  all  provisions  of  the  contract  and  for 
furnishing  specified  supplies  and  services  which  meet  all 
the  requirements  of  the  contract.  ISO  does  not  have  this 
statement . 

•  ISO  provides  for  tailoring  of  specific  contractual 
situations.  MIL-Q-9858  does  not  have  such  a  provision. 

•  ISO  states  that  it  is  primarily  aimed  at  preventing 
nonconformity.  MIL-Q-9858  does  not  state  this  primary 
aim. 

•  ISO  applies  when  the  contract  specifically  requires  design 
effort  and  the  product  requirements  are  stated  principally 
in  performance  terms  or  they  need  to  be  established.  MIL- 
Q-9858  applies  to  all  supplies  or  services  when  referenced 
in  the  contract. 

•  MIL-Q-9858  requires  the  quality  program,  including 
procedures,  processes  and  product,  to  be  subject  to  review 
by  the  Government  Representative,  to  have  access  to  and 
the  right  to  review  and  evaluate  all  aspects  of  the 
contractor's  quality  program.  ISO  does  contain  provisions 
for  purchaser  review  and  evaluation  except  where  agreed  to 
contractually,  (paragraph  4.16) 


The  OSD ( P&L )  comment  concerning  the  prevention  of 
nonconformity  as  a  goal  for  ISO,  while  it  is  not  for  MIL-Q, 
addresses  the  philosophical  difference  between  the  two 
standards.  ISO  is  concentrating  on  the  development  of  an 
organization  that  is  set  on  improvement  and  the  prevention  of 
nonconforming  material;  MIL-Q  concentrates  only  on  inspections 
and  detection  of  nonconforming  material.  The  last  comment 
regarding  access  to  evaluate  the  quality  assurance  system  of 
a  company  is  really  a  legal  issue  that  has  been  buried  in  the 
MIL-STD,  instead  of  being  addressed  in  contractual  language 
where  it  appropriately  belongs. 
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2.  Quality  Systems  Requirements/Quality  Policy. 

This  is  a  paragraph  in  ISO  9001  to  which  MIL-Q-9858 
has  no  equivalent.  The  OSD(P&L)  comments  are: 

«  ISO  requires  the  supplier's  management  to  define  and 
document  its  commitment  to  quality.  MIL-Q-9858  does  not 
require  this. 

This  provision  of  ISO  9001  clearly  delineates  the 
heart  of  the  difference  between  it  and  MIL-Q-9858.  ISO  is 
designed  to  create  an  organization  that  is  committed  to 
continuous  improvement  of  quality,  and  that  commitment  has  to 
come  from  the  top  echelons  of  management.  MIL-Q-9858  is  an 
attempt  to  put  quality  into  the  process/product  through 
continuous  inspection. 

3.  Organization/Management. 

The  organization  and  responsibility  within  the  Quality 
Assurance  department  of  a  company  is  addressed  in  both 
standards.  OSD(P&L)  comments  are: 


•  ISO  requires  the  responsibility,  authority,  and  the 
interrelation  of  all  personnel  who  manage,  perform,  and 
verify  work  affecting  quality  be  defined.  MIL-Q-9858 
requires  that  the  responsibility  and  authority  of 
personnel  performing  quality  functions  be  defined, 
(paragraph  3.1) 

•  ISO  requires  the  supplier  to  assign  trained  personnel  for 
verification  activities.  MIL-Q-9858  does  not  specifically 
require  the  assignment  of  trained  personnel. 

•  ISO  requires  that  verification,  including  inspection, 
test,  and  monitoring,  be  carried  out  by  personnel, 
independent  of  those  having  direct  responsibility  for  the 
work  being  performed.  MIL-Q-9858  requires  that  personnel 
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performing  quality  functions  shall  have  the  organizational 
freedom  to  identify  and  evaluate  quality  problems, 
(paragraph  3.1) 

•  ISO  requires  that  a  management  representative  be 
responsible  for  ensuring  that  the  requirements  of  the 
quality  system  are  implemented  and  maintained.  MIL-Q-9858 
indicates  that  the  fulfillment  of  the  quality  program 
requirements  is  not  the  responsibility  of  any  single 
contractor's  organization,  function  or  person. 


The  ISO  standards  are  written  to  improve  the  quality 
assurance  system  within  a  company.  An  important  element  of 
this  program  is  the  initiation  of  a  program  of  continuous 
improvement.  The  concept  of  continuous  improvement  requires 
that  people  be  trained  and  qualified  to  operate  and  maintain 
the  quality  system  described  by  ISO  9000  standards.  The  MIL-Q 
is  less  stringent  concerning  employee  qualification,  because 
it  has  an  inherent  attitude  that  more  inspection  will  reveal 
the  defects  and  thus  overcome  the  failures  that  may  exist  in 
the  level  of  training  within  the  company's  quality  assurance 
system. 

4.  Quality  System/ Program. 

These  sections  of  the  standards  deal  with  the 
requirements  for  establisning  a  quality  assurance  system 
within  the  company.  ISO  requires  the  establishment  of  a 
system  that  covers  all  phases  of  the  production  process,  while 
MIL-Q-9858  is  written  in  terms  of  a  program  that  is  an  end  in 
and  of  itself.  OSD(PScL)  comments  are: 
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•  ISO  provides  that  consideration  needs  to  be  given  for  the 
preparation  of  a  quality  plan  (defined  in  ISO  8402  as  a 
document  setting  out  the  specific  quality  practices, 
resources  and  sequences  of  activities  relevant  to  a 
particular  product,  service,  contract  or  project)  .  MIL-Q- 
9858  does  not  have  any  provision  for  a  quality  plan. 

•  ISO  provides  that  consideration  needs  to  be  given  for  the 
compatibility  of  the  design,  the  production  process, 
installation,  and  the  applicable  documentation.  MIL-Q- 
9858  does  not  provide  for  the  compatibility  of  the  design 
and  production  process. 


5.  Contract  Review/Design,  Control / Product , 

Identification  and  Traceability. 

These  are  three  topics  covered  by  ISO  and  not  covered 
in  MIL-Q-9858.  The  OSD(P&L)  comments  are: 


•  ISO  requires  the  supplier  to  review  each  contract  and 
maintain  records  of  these  reviews.  MIL-Q-9858  requires 
the  contractor  to  conduct  a  complete  review  of  the 
requirements  of  the  contract.  Records  of  these  reviews 
are  not  specifically  required. 

■»  ISO  devotes  an  entire  section  to  design  control.  MIL-Q- 
9858  does  not  address  design  control. 

•  ISO  requires  the  identification  of  the  product  from 
applicable  drawings,  specifications,  or  other  documents 
during  all  stages  of  production,  delivery,  and 
installation.  Such  identification  is  not  required  in  MIL- 
Q-9858. 


In  both  these  cases,  ISO  is  covering  the  entire 
process  that  a  contractor  must  complete  to  achieve  an 
acceptable  end  product.  It  attempts  to  avoid  an  unacceptable 
end  product  by  ensuring  that  the  entire  production  process  is 
under  control.  The  MIL-Q  relies  on  an  inspection  program  to 
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detect  and  ultimately  remove  defective  products  after  they 
have  been  produced. 

6.  Document  Control/Drawings,  Documentation  and  Changes . 

The  procedures  prescribed  in  the  standards  to  control 
changes  to  the  technical  data  package  are  fundamentally 
different.  MIL-Q  advocates  a  broader  approach  in  evaluating 
the  adequacy  of  records  keeping  with  regard  to  engineering 
changes,  while  ISO  has  specific  steps  to  be  followed  in  this 
area.  The  OSD(P&L)  comments  reflect  this  difference: 


•  MIL-Q-9858  requires  a  procedure  for  the  evaluation  of  the 
adequacy  of  design  drawings  and  specifications  in  relation 
to  standard  engineering  and  design  practices,  and  with 
respect  to  the  design  and  purpose  of  the  product.  This 
requirement  is  not  in  ISO. 

•  ISO  requires  that  a  master  list  or  equivalent  document 
control  be  established.  ISO  also  requires  that  documents 
be  re-issued  after  a  practical  number  of  changes  have  been 
made.  These  requirements  are  not  in  MIL-Q-9858. 

•  MIL-Q-9858  extends  the  contractor's  responsibility  to 
drawings  and  changes  provided  by  subcontractors  and 
vendors.  This  requirement  is  not  in  ISO. 


The  ISO  9001  standard  gives  more  detailed  guidance  in 
establishing  control  over  the  technical  data  package.  The 
MIL-Q-9858  leaves  this  area  up  to  the  contractor's  established 
systems.  It  tries  to  use  an  industry  best-practices  approach 
to  document  control,  based  on  the  theory  that  there  is  such  a 
wide  discretionary  range  in  approaching  this  problem  that 
whatever  system  the  contractor  is  using  is  acceptable,  as  long 
as  it  is  close  to  industry  practices. 


36 


7.  Purchasing  Control. 

OSD(P&L)  comments: 


•  ISO  requires  the  establishment  and  maintenance  of  records 
of  acceptable  subcontractors.  Such  records  are  not 
required  in  MIL-Q-9858. 

•  MIL-Q-9858  provides  for  inspection  by  the  Government  at 
the  subcontractor's  plant.  ISO  provides  for  the  purchaser 
or  his  representative  to  verify  at  source,  when  specified 
in  the  contract,  that  the  purchased  product  conforms  to 
specified  requirements.  (para  4.6.4) 

•  MIL-Q-9858  requires  the  use  of  test  reports,  inspection 
records,  certificates  and  other  suitable  evidence  relating 
to  the  subcontractor's  control  of  quality.  MIL-Q-9858 
requires  the  contractor  to  have  procedures  for  the  early 
information  feedback  and  correction  of  subcontractor 
nonconformances.  This  is  not  required  in  ISO. 

•  MIL-Q-9858  includes  requirements  for  chemical  and  physical 
testing  and  recording  in  connection  with  the  purchase  of 
raw  materials  by  suppliers.  Such  testing  is  not  required 
in  ISO. 


The  ISO  standards  reflect  an  attitude  that  contractors 
should  use  a  "best  value"  approach  in  selecting  their 
subcontractors.  The  definition  of  "best  value"  is  the 
attainment  of  maximum  benefit  from  money  spent  in  terms  of 
satisfying  requirements  of  the  contract.  Thus,  the 

establishment  of  long-term  relationships  between  contractor 
and  supplier  is  not  looked  upon  in  Europe  as  being  "collusive" 
as  it  frequently  is  in  the  U.S.  DoD  stresses  use  of  the 
lowest  bidder  for  subcontract  work,  creating  a  barrier  to 
development  of  long-term  relationships.  If  a  subcontractor  is 
not  the  lowest  bidder  for  future  business,  the  prime 
contractor  is  obliged  to  select  the  lowest  cost  proposal. 
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regardless  of  past  performance  considerations.  The  value  of 
a  proven  subcontractor  carries  little  weight  when  prime 
contractors  are  evaluated  for  control  of  subcontracts  by  DoD. 
The  MIL-Q-9858A  is  relying  on  incoming  inspection  to  detect 
defective  products,  rather  than  trying  to  establish  a  rapport 
with  a  subcontractor  to  eliminate  substandard  material. 

8.  Purchaser  Supplied  Product /Government  Property. 

ISO  9001  has  a  brief  three-sentence  paragraph  that 
requires  a  procedure  be  set  for  receipt,  storage  and  reporting 
of  problems 

regarding  purchaser  supplied  material.  There  are  no  specific 
directions  as  to  protection  and  disposition  of  such  material. 
Meanwhile,  MIL-Q-9858A  has  three  lengthy  paragraphs  that 
explain  what  action  will  be  taken  if  problems  arise  with 
Government  Furnished  Property  (GFP) .  MIL-Q-9858A  attempts  to 

cover  all  contingencies  of  this  situation.  The  OSD(P&L) 
comment  is: 

•  MIL-Q-9858  requires  the  contractor  to  provide  more 
specific  protection  of  Government -furnished  material.  The 
control  of  purchaser  supplied  product  in  ISO  is  more 
general . 

9.  Process  Control/Work  Instructions. 

These  sections  of  the  standards  discuss  how  a  company 
documents  the  division  of  labor  within  the  manufacturing 
process.  In  essence  it  is  how  the  bill  of  material  and  labor 
sheets  are  written.  OSD(P&L)  comments: 
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•  ISO  requires  work  instructions  defining  the  manner  of 
production  and  installation  only.  MIL-Q-9858  requires 
work  instructions  for  purchasing,  handling,  machining, 
assembling,  fabricating,  processing,  inspecting,  testing, 
modification,  installation,  and  many  other  treatments  of 
product,  facilities,  standards,  or  equipment. 


The  OSD(PScL)  comment  on  this  section  of  the  standards 
would  lead  one  to  conclude  that  it  covers  less  than  MIL-Q- 
9858.  In  reality,  ISO  9001  encompasses  more  than  MIL-Q-9858 
in  this  area  because  it  focuses  on  covering  the  entire 
manufacturing  process  with  controlled  written  procedures.  The 
MIL-Q-9858  states  the  requirements  for  this  area  in  terms  of 
"all  work  affecting  quality."  Since  ISO  covers  the  entire 
process,  i.e.,  all  work  affecting  quality,  both  standards  are 
identical  in  intent. 

10.  Inspection  and  Testing/Manufacturing  Control. 

ISO  9001  concentrates  on  establishing  a  process  that 
assures  an  acceptable  incoming  product  from  suppliers.  MIL-Q- 
9858  relies  on  inspection  to  single  out  nonconforming 
material.  OSD(P&L)  comments: 

•  MIL-Q-9858  requires  raw  materials  to  conform  to  the 
applicable  physical,  chemical,  and  other  technical 
requirements.  Laboratory  testing  shall  be  employed  as 
necessary.  ISO  does  not  require  such  testing. 


11.  Final  Inspection  and  Test. 

The  OSD { P&L )  comment  in  thic  area  is: 


•  MIL-Q-9858  requires  reporting  to  designers  any  unusual 
difficulties,  deficiencies  or  questionable  conditions 
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during  final  inspection  and  testing.  ISO  does  not  require 
this  feedback. 


The  OSD  comment  in  this  area  fails  to  view  ISO  9001  in 
its  entirety.  ISO  requires  more  formalization  of 

feedback/corrective  action  procedures  than  does  MIL-Q-9858. 
It  simply  does  not  list  those  corrective  action  steps  at  this 
point  in  the  standard.  The  philosophy  that  supports  ISO  9001 
would  say  that  nonconformance  at  final  inspection  means  there 
was  a  failure  of  the  quality  assurance 

system  because  it  failed  to  prevent  the  nonconformance  in  the 
first  place. 

12 .  Measuring  and  Test  Equipment . 

The  OSD ( P&L)  comments: 

•  MIL-Q-9858  is  more  specific  and  elaborate  in  the  area  of 
measuring  and  test  equipment  by  requiring  conformity  with 
MIL-C-45662. 

•  MIL-Q-9858  requires  the  contractor  to  make  his  gauges, 
measuring  and  testing  devices  available  to  the  Government. 
ISO  does  not  have  this  requirement. 

The  first  comment  listed  above  seems  to  be  a  moot 
point.  If  a  company  is  using  ISO  9001,  it  can  supplement  it 
with  other  standards.  For  example,  a  contract  written  using 
ISO  9001  as  a  requirement,  still  can  designate  MIL-C-45662 
(calibration)  as  a  required  technical  specification. 
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The  second  comment  listed  above  concerning  Government 
access  is  another  example  of  where  MIL-Q-9858  was  used  as  a 
vehicle  for  contractual  requirements  that  belong  elsewhere  in 
a  contract,  not  in  the  quality  assurance  standard. 


13.  Control  of  Nonconforming  Material. 

In  the  area  of  the  nonconforming  material,  ISO  9001 
and  MIL-Q-9858A  take  different  approaches  to  resolving  rework, 
due  to  the  difference  in  the  philosophy  of  the  standards.  ISO 
views  rework  as  a  source  of  information  back  to  the  process 
that  improvement  is  possible.  Thus  ISO  specifies  that  rework 
effort  is  to  undergo  the  same  inspection  as  the  original 
effort.  MIL-Q-9858A  calls  for  separate  procedures  to  resolve 
rework,  thus  removing  a  feedback  loop  back  to  the  original 
production  process.  The  OSD(P&L)  comments  are: 


•  MIL-Q-9858  states  that  the  acceptance  of  nonconforming 
supplies  is  a  prerogative  of,  and  shall  be  as  prescribed 
by,  the  Government.  ISO  does  not  have  an  equivalent 
statement.  MIL-Q-9858  requires  that  repair  or  rework  be  in 
accordance  with  documented  procedures  acceptable  to  the 
Government.  Procedures  for  repair  or  rework  are  not 
required  by  ISO.  ISO  requires  that  repaired  and  reworked 
product  be  re-inspected  in  accordance  with  documented 
procedures.  MIL-Q-9858  does  not  require  re-inspection. 

•  MIL-Q-9858  requires  data  associated  with  the  costs  and 
losses,  in  connection  with  scrap  and  rework,  be  made  known 
to  the  Government.  This  is  not  required  in  ISO. 
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14.  Handling,  Storage,  Packaging  and  Delivery. 

The  intent  of  both  standards  is  the  same  in  this  area; 
the  comment  below  seems  to  this  researcher  to  be  one  of 
semantics : 


•  MIL-Q-9858  requires  periodic  inspection  for  the  prevention 
of  deterioration  or  damage  in  storage.  ISO  requires 
periodic  assessment  of  product  in  stock  to  detect  (not 
prevent)  deterioration. 


15.  Quality  Records. 

Because  of  the  emphasis  in  MIL-Q-9858  on  final  product 
inspection,  the  MIL-STD  sets  specific  requirements  for 
records-  keeping  regarding  final  inspection.  ISO  relies  on 
feedback  and  continual  improvement,  leaving  open  the  exact 
form  of  the  data,  so  long  as  it  is  brought  to  the  attention  of 
personnel  responsible  for  the  area  being  tested.  The  OSD(P&L) 
comments  are: 


•  MIL-Q-9858  requires  records  to  indicate  the  nature  of  the 
observations,  together  with  the  number  of  observations 
made  and  the  number  and  type  of  deficiencies  found.  ISO 
does  not  specify  the  contents  of  the  records. 

•  MIL-Q-9858  requires  records  to  be  used  as  a  basis  for 
management  action.  ISO  does  not  have  this  requirement. 

•  MIL-Q-9858  does  not  have  a  requirement  for  internal 
quality  audits  as  required  in  ISO. 


16.  Training  and  Servicing. 

These  are  two  areas  not  covered  by  MIL-Q-9858. 
OSD ( P&L )  comment  s : 
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•  ISO  requires  a  procedure  for  the  identification  and 
provision  of  training  for  all  personnel  performing 
activities  affecting  quality.  MIL-Q-9858  does  not  have 
this  requirement. 

•  ISO  requires  procedures  to  assure  that  servicing 
requirements  are  met.  MIL-Q-9858  does  not  cover 
servicing . 


17.  Statistical  Techniques. 

The  ISO  Standards  require  statistical  methods  of 
control  be  applied  to  all  processes  covered  by  the  standards. 
MIL-Q,  on  the  other  hand,  uses  sampling  to  conduct 
inspections.  Again  it  is  the  fundamental  atticude  of  either 
controlling  the  process  [ISO]  or  inspecting  the  end  product 
and  removing  unacceptable  material  [MIL-Q].  OSD(P&L) 
comments  are: 


•  ISO  requires  the  use  of  statistical  techniques  for  process 
capability  and  for  product  characteristics.  MIL-Q-9858 
does  not  have  provision  for  such  statistical  techniques. 
MIL-Q-9858  allows  the  contractor  to  use  sampling 
procedures  for  product  acceptance.  ISO  does  not 
specifically  allow  sampling. 


18.  Costs  Related  to  Quality. 

This  is  the  one  major  area  found  in  MIL-Q-9858  which  is 
totally  absent  in  ISO.  OSD(P&L)  comments: 


•  MIL-Q-9858  requires  the  contractor  to  maintain  and  use 
quality  cost  data.  ISO  does  not  require  quality  cost 
data . 
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G.  SUMMARY 


An  armchair  comparison  of  ISO  9000  to  Military  Standards 
leaves  this  researcher  with  the  impression  that  a  change  to 
ISO  would  not  be  difficult,  and  the  resulting  Quality 
Assurance  program  changes  would  aid  in  improving  quality  and 
reducing  costs.  A  survey  of  contractors,  described  in  the 
next  chapter,  reinforces  this  impression. 
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IV.  DATA  ANALYSIS /INTERPRETATION 


A.  INTRODUCTION 

The  data  presented  in  this  study  were  gathered  through  a 
sampling  of  24  managers  or  directors  of  quality  assurance 
departments  from  companies  that  are  ISO  9000  registered  and 
were  selected  on  the  likelihood  that  they  were  familiar  with 
MIL-STDS.  Each  survey  was  conducted  by  telephone  and  lasted 
from  20  to  45  minutes.  The  respondents  were  encouraged  to 
answer  freely  on  a  nonattribution  basis.  The  aggregate 
listing  of  respondents  is  presented  in  Appendix  A. 

In  order  to  reach  companies  primarily  concerned  with  DoD 
contracting,  only  those  companies  listed  in  the  following 
categories  were  contacted:  [Ref  14] 

•  Rubber  and  Miscellaneous  Plastics  Products 

•  Primary  Metal  Industries 

•  Fabricated  Metal  Products 

•  Industrial  and  Commercial  Machinery  and  Computer  Equipment 

•  Electronic  and  Other  Electrical  Equipment 

•  Measuring,  Analyzing  and  Controlling  Instruments 

•  Wholesale  Trade  -  Durable  Goods 

The  researcher  was  interested  only  in  contacting  those 
companies  actively  involved  in  DoD  contracts  and  that  were 
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aware  of  ISO  9000  Standards.  As  a  result,  the  data  are 
heavily  biased  toward  positive  comments  regarding  Icr  ^000 
because  the  firms  contacted  have  already  demonstrated  a  strong 
commitment  to  ISO  9000  by  completing  the  registration  process. 

Of  the  25  companies  interviewed,  20  of  them  are  registered 
to  the  ISO  9001  standard  and  five  are  registered  to  the  ISO 
9002  standard.  All  of  the  respondents  were  managers  or 
directors  of  quality  assurance  departments  within  their 
companies  and  all  had  recent  experience  with  the  registration 
process.  In  many  cases  the  person  interviewed  was  responsible 
for  planning  and  achieving  ISO  registration,  and  was  thus  very 
familiar  with  the  problems  faced  by  a  company  attempting  to 
achieve  registration. 

B .  THE  RESPONSES 

1.  Question  One. 

Does  your  company  currently,  or  has  it  in  the  past, 
performed  DoD  contracts  that  specified  compliance  with  MIL-Q- 
9858A  or  MIL-I-45208? 

a.  Discussion 

The  purpose  of  this  question  was  to  determine  if 
the  person  interviewed  was  familiar  with  MIL-Q-9858  or  MIL-I- 
45208  and  if  the  company  fell  into  the  target  group  of  DoD 
contractors . 
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b.  Analysis 


In  24  of  the  25  companies  interviewed,  the  person 
contacted  was  familiar  with  MIL-Q-9858.  This  familiarity 
included  compliance  with  MIL-STDS  on  current  or  previous 
contracts  and  a  thorough  knowledge  of  the  quality  assurance 
system  their  company  designed  in  order  to  comply  with  MIL-Q- 
9858  or  MIL-I-45208.  Of  this  group,  only  16  were  actively 
involved  in  DoD  contracts  at  the  time  the  interview  was 
conducted . 

2.  Question  Two. 

Was  your  company  aware  of  DoD's  policy  to  use  ISO  9000 
in  contracts,  and  if  so,  was  it  a  factor  in  your  company's 
decision  to  attain  ISO  registration? 
a.  Discussion 

This  question  was  asked  in  order  to  get  a  feel  for 
the  level  of  awareness  in  industry  with  regard  to  ISO 
implementation  by  DoD.  There  were  eight  people  interviewed 
who  knew  that  DoD  was  reviewing  the  use  of  ISO  9000  in  DoD 
contracts,  and  only  three  who  knew  that  DoD  was  contemplating 
the  replacement  of  MIL-Q-9858  with  ISO  9001.  The  official 
position,  as  expressed  by  the  Assistant  Secretary  of  Defense 
in  1989,  was  completely  new  to  all  25  respondents.  When  asked 
if  DoD  policy  on  ISO  9000  was  a  factor  in  getting  registered, 
only  one  person  said  it  was,  and  he  quickly  stated  that  it  was 
only  a  minor  factor. 
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b .  Analysis 

The  survey  made  it  clear  that  the  driving  force 
for  a  firm  to  become  registered  is  centered  on  remaining 
competitive  in  the  global  marketplace,  in  order  to  sell  to  the 
European  Community  after  1992.  None  of  the  comments  indicated 
that  consideration  of  future  DoD  business  was  a  factor  in  the 
decisions.  The  decision  to  become  ISO  9000  registered  is 
obviously  based  on  a  desire  for  additional  international 
sales.  Some  of  the  comments  regarding  this  international 
marketplace  are  paraphrased  below: 


•  We  went  after  9001,  [because]  we  are  a  U.S.  company  that 
does  a  lot  of  work  in  Europe  and  [we]  export  a  lot  to 
Europe.  There  is  a  theoretical  requirement  that  by 
December  of  '92,  ISO  9000  will  be  required  to  do  business 
within  the  European  community.  So  ISO  9000  is  driven  by 
the  commercial  customers  in  Europe. 

•  [ISO  9000]  is  an  advantage  in  the  international  [markets] . 

But  it  [ISO  9000]  will  become  it  [quality  assurance]  in 
the  U.S.  in  very  short  order,  I  can  see  it.  Our 

competition  is  scrambling  right  now  because  we  have  it  and 
they  don't.  And  we  are  going  to  just  start  flaunting  it 
from  a  marketing  point  of  view. 

•  I  think  the  primary  impetus  of  this  activity  is  that  we 

have  a  high  percentage  of  international  (especially 
European)  sales.  And  it  is  part  of  the  Europe  '92 

initiative.  We  felt  this  was  important. 

•  A  number  of  years  ago  we  started  to  pick  up  on  this  kind 

of  requirement  in  the  international  community.  We  deal 
with  a  lot  of  major  contractors  --  international 
contractors  --  and  we  started  to  see  a  lot  more  stringent 
quality  requirements  come  through  with  regard  to 
contracts.  So  we  looked  to  see  what  was  out  there,  and 

found  that  this  standard  had  been  revised  in  '87  and  that 

a  number  of  countries  (40-some-odd  of  them)  had  also 
pretty  much  duplicated  their  national  specs,  such  as  our 


48 


ANSI  ASQ  C90,  so  at  any  rate,  that  is  why  we  started 
considering  it  and  that's  why  we  did  it. 

•  ...  by  virtue  of  us  trying  to  maintain  the  ability  to  sell 
in  Europe.  So  the  influence  from  the  European  Economic 
Community  had  influence  as  well. 

•  ...  ISO  standard  has  been  applicable  since  '87  and  [the 
motive]  was  more  that;  if  you  intended  doing  business  in 
the  European  Economic  Community,  you  should  address  that 
criterion . 

•  We  became  ISO  9000  qualified  for  a  specific  commercial 
sale.  A  customer  of  ours  was  looking  for  a  marine 
industrial  engine.  It  was  a  European  customer  and 
required  that  we  be  ISO  9000  certified. 


In  addition  to  the  marketing  drive  to  remain 
competitive,  the  researcher  also  discovered  that,  for  some 
companies,  ISO  9000  is  simply  their  next  step  down  a  road  to 
continuous  improvement  and  ultimately  Total  Quality 
Management.  This  situation  is  illustrated  by  the  following 
paraphrased  comments: 


•  ...  we  were  in  the  throes  of,  and  still  are  in  the  throes 

of,  finalizing  implementation  of  a  total  quality 
management  system.  Well,  we  have  about  10  different 
subsections  in  our  quality  management  program.  And  there 
are  63  different  clauses  that  we  try  to  follow  and  about 
80  percent  are  required  by  ISO.  So  getting  the  ISO 

registration  brought  us  to  better  than  80  percent  into 
implementing  our  total  quality  management  system.  It  gave 
us  a  nice  independent  third-party  evaluation  of  where  we 
were  in  implementing  our  own  quality  program.  So  that 
fortified  our  whole  quality  organization,  made  us  feel 
pretty  good.  We  were  further  than  we  realized. 

•  We  originally  started  with  what  they  call  QMI  --  it's  a 
division  of  Canadian  Standards  Association.  It's  like  the 
ASME  with  Canada.  Now  they're  going  to  all  QMI  or  now  ISO 
9000,  but  some  three  years  ago  we  went  with  the  QMI  and 
passed  this  Canadian  audit,  and  it  was  just  a  very  minute 
revising  of  our  manual  to  get  into  the  ISO  9000. 


It  should  be  remembered  that  this  group  of 
companies  is  in  a  select  minority.  There  are  only  225  U.S. 
firms  registered  to  the  ISO  9000  Standards  and  they  are  the 
very  early  leaders  in  a  growing  movement  toward  international 
standardization  of  quality.  The  firms  contacted  were  selected 
on  the  likelihood  that  they  were  involved  in  high  technology 
fields  of  production.  The  ISO  9000  Standards  are  more  easily 
implemented  in  the  manufacturing  environment,  and  return  the 
most  benefit  to  firms  engaged  in  high  technology  fields 
requiring  a  high  degree  of  quality  control  in  their  production 
processes . 

3.  Question  Three. 

I£  DoD  puts  ISO  9000  requirements  in  a  contract,  would 
your  company  also  agree  to  use  your  ISO  registrar  as  an 
arbitrator  of  disputes  between  your  firm  and  DoD? 
a .  Discussion 

This  question  was  designed  to  support  or  refute  a 
recommendation  that  will  be  discussed  in  the  next  chapter  of 
this  thesis.  It  was  explained  during  the  interview  that  the 
ISO  registrar  would  be  contacted  only  as  a  last  resort  in  the 
event  of  a  deadlocked  position  arising  between  the  Government 
and  contractor.  In  addition,  the  loser  would  pay  the  expense 
of  having  retained  the  arbitrator.  This  payment  provision  was 
added  to  ease  concerns  that  the  Government  would  stonewall  a 
contractor  and  drag  all  quality  disputes  before  a  registrar, 
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leaving  the  bill  for  registrar  services  to  the  contractor  for 
payment.  This  entire  suggestion  of  incorporating  the  ISO 
registrar  as  a  last-resort  arbitrator  was  intended  to  provide 
a  new  method  of  handling  quality  disputes  with  contractors,  in 
such  a  manner  as  to  reduce  cost  and  delay  factors  associated 
with  some  of  the  really  difficult  disputes  between  DoD  and 
contractors  over  quality  issues. 
b.  Analysis 

All  25  respondents  to  this  survey  were 
enthusiastic  supporters  of  implementing  ISO  9000  Standards  in 
DoD  contracting,  and  for  using  registration  as  certification 
that  a  contractor's  quality  system  was  satisfactory  for  DoD 
contract  purposes.  Thus  pre-award  survey  audits,  to  determine 
if  a  contractor  has  an  acceptable  quality  system  in  place,  are 
no  longer  performed.  If  a  contractor  has  current  ISO  9000 
registration,  his  quality  system  is  automatically  acceptable 
on  a  pre-award  survey. 

The  issue  of  arbitration  was  suggested  by  the 
researcher  as  a  means  to  achieve  even  more  benefit  from  a 
contractor  being  ISO  9000  registered.  Arbitration  becomes 
applicable  in  those  situations  where  a  Government  Quality 
Assurance  Representative  (QAR)  believes  the  contractor  is  not 
adhering  to  the  quality  policies  and  system  used  during  the 
registration  process.  A  complaint  filed  by  a  QAR  with  a 
company's  registrar  would  jeopardize  the  company's  ISO  9000 
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registration,  and  any  commercial  business  requiring  that 
registration.  The  final  tally  on  the  arbitration  issue  showed 
14  in  favor  of  it,  eight  opposed,  and  two  interviewees  who 
refused  to  commit  either  way.  The  main  reason  for  opposing 
the  arbitration  appeared  to  be  that  it  was  unnecessary  because 
disputes  should  be  settled  before  they  reach  arbitration.  A 
sample  of  comments  along  these  lines  is  paraphrased  below: 


•  I  don't  think  it  should  be  necessary  to  bring  a  third 
party  into  the  picture.  It  should  be  something  that  we  as 
the  company  and  the  Government  should  be  able  to  resolve 
between  us . 

•  Any  company,  I  think,  would  not  want  to  have  an  outside 
party  control  their  destiny.  If  there  is  a  disagreement, 
they  want  to  have  the  ability  to  negotiate  it  without 
having  someone  else  come  in  and  say  you're  right  or  wrong. 


•  I  really  rather  see  it  worked  out  between  the  supplier  and 
the  customer  and  come  to  a  consensus,  rather  than  bring  in 
a  third  party. 

•  ...  if  you  accept  the  thesis  that  anybody  that  comes  in 
should  be  thoroughly  knowledgeable  and  professional  in  the 
auditing  art  or  science,  or  both,  then  disputes  that  must 
be  taken  to  arbitration  should  be  almost  nonexistent.  I 
am  a  certified  quality  auditor,  and  I  feel  I  should  be 
able  to  resolve  issues  like  that  with  a  qualified  auditor 
who  comes  in  my  building.  I  know  his  business  as  well  as 
he  does  and  we  ought  to  be  able  to  resolve  those  issues; 
if  not,  then  I  feel  comfortable  that  I  wouldn't  take 
something  to  arbitration  unless  I  really  felt  I  was  on 
secure  ground. 

•  ...  basically,  nobody  likes  to  air  his  dirty  laundry 
where  it  might  get  back  to  the  customer.  It  could  get  to 
the  competition  somehow,  somebody,  might  just  pass  it 
along.  I  always  worry  about  that  sort  of  thing  getting 
into  the  so-called  wrong  hands,  whatever  that  means. 

•  We  don't  want  disputes  to  reach  that  level.  You  are 
really  asking  for  an  independent  mediator. 
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•  I  would  think  that  before  [arbitration]  I  would  prefer  to 
escalate  it  internally.  Have  more  senior  quality  officers 
be  involved,  rather  than  going  to  external  arbitration. 

•  Well,  by  and  large,  I  think  the  military  side  and  the 
commercial  side,  the  third  party  side,  of  auditing  are 
pretty  much  in  agreement  on  these  things.  After  20  years, 
I  haven't  seen  things  that  couldn't  be  thought  through  and 
worked  out  at  the  plant  level. 

•  I  think  the  first  move  should  not  be  going  to  UL.  The 
first  move  should  be  coming  to  the  contractor,  giving  the 
contractor  an  opportunity  to  correct  the  oversight.  If 
there  is  dispute  on  interpretation  of  what  the  program  is 
requiring,  that  should  be  resolved  before  anything  else 
occurs . 


The  predominate  response  to  the  question  of  using 
arbitration  was  that  it  should  be  unnecessary.  Yet,  when 
asked,  many  of  the  people  interviewed  agreed  that  arbitration 
as  a  last  resort  was  a  good  idea.  The  Contracting  Officer  who 
puts  ISO  9000  into  a  contract  may  want  to  consider  the  use  of 
ISO  9000  registrars  as  arbitrators.  The  idea  of  arbitration 
is  particularly  appealing  when  dealing  with  contractors  or 
industries  that  have  a  history  of  sending  quality  problems  up 
to  higher  levels  of  management  for  resolution.  The  data 
indicate  that  use  of  registrars  as  arbitrators  would  not 
create  a  prohibitive  barrier  for  ISO  9000  registered 
contractors  bidding  on  a  contract. 

In  addition,  the  ISO  9000  philosophy,  which  says 
that  customers  should  receive  the  quality  of  product  they 
desire,  provides  additional  motivation  to  avoid  disputes 
between  customer  and  contractor.  The  probability  of  disputes 
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is  reduced  by  adopting  fully  the  ISO  Standards,  regardless  of 
any  arbitration  clause  that  might  be  a  part  of  the  contract. 

The  follow-up  discussion  concerning  the  use  of 
registrars  as  arbitrators  revealed  insights  into  two  more 
systemic  problems  regarding  quality  assurance  that  ISO  9000 
has  the  potential  to  resolve  or  alleviate.  These  potential 
problems  include: 

(1)  Adversarial  attitudes .  There  were  repeated 
comments  that  implementation  of  arbitrators  would  increase  the 
adversarial  posturing  that  now  exists.  Samples  of  these 
comments  are  paraphrased: 


•  It  would  be  a  concern,  just  because  it  throws  somebody 
else  into  the  equation  and  gets  an  adversarial 
relationship  going  instead  of  the  teamwork  atmosphere  and 
the  partnership  atmosphere  that  ought  to  be  going  between 
the  customer  and  supplier.  There's  been  too  much  of  that 
in  the  defense  portion;  I  saw  it  over  at  ...  too  much  and 
there's  too  much  of  it  in  non-defense  anyway.  But  it  is 
getting  better. 


•  I  think  the  registrar  could  be  used  between  the  two 
participants  where  they  contact  the  registrar  and  get  the 
registrar's  point  of  view  on  it.  But  as  an  official  legal 
arbitration  device,  I'd  just  hate  to  see  that  happen 
because  it  gets  the  adversarial  relationship  going  again. 
It's  us  versus  you.  Let's  call  this  third  party  to  decide 
between  us,  and  they  decide,  and  now  I'm  angry  because 
they  decided  against  me,  and  I  resent  that  . . . 


The  discussion  of  registrars  as  arbitrators 
revealed  just  how  hostile  the  relationship  between  DoD  and 
defense  contractors  can  become.  The  interviewees  are  all 
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experienced  quality  assurance  managers  whose  comments  show  a 
sensitivity  toward  any  idea  or  proposal  that  has  the  potential 
of  increasing  the  adversarial  attitudes  between  DoD  and 
defense  contractors. 

(2)  Quality  of  Government  versus  ISO  auditors .  A 
second  topic  that  arose  frequently  during  discussions  of 
registrars  as  arbitrators  dealt  with  comparisons  of  ISO 
auditors  to  Government  auditors.  These  comparisons  surfaced 
when  interviewees  were  discussing  the  advantages  of  using  ISO 
registrars  to  settle  quality  disputes.  A  sampling  of  these 
comments  is  provided  below: 


•  I  gave  a  paper  at  a  symposium  awhile  ago  talking  about 

9858  and  that  was  one  of  the  biggest  [topics]  ;  ...  it  was 

to  a  mixed  group  of  military  and  civilian-type  people  . . . 
the  thing  that  I  harangued  on  was  the  qualification  of  the 
auditors  that  came  into  my  plant.  I  got  all  sorts  of 
grief,  because  I  said  that  these  people  aren't 
professional  ...  they're  misfits,  malcontents,  and 
otherwise  unqualified  individuals  who  couldn't  forge  a 
career  for  themselves  in  other  places  and  they  got  dumped 
into  the  quality  area.  I  really  resent  that.  I  resent 
those  kinds  of  people  coming  into  rny  plant  and  casting  my 
destiny.  They  focus  on  the  minutia  and  the  wherefores  and 
the  whereases  and  they  do  not  understand  the  major  issues 
involved  to  assure  the  integrity  of  the  product  being 
delivered  to  them. 

•  Right  now  I  am  more  impressed  with  the  auditor  from  the 
registrar  than  I  have  been  from  the  Government  folks. 

•  What  we  find  is  that  ISO  auditors  dig  much  deeper  in 
certain  areas  because  they  are  looking  at  the  whole 
business  cycle  as  opposed  to  what  Government  auditors  do. 

•  The  other  benefit  that  you  can  include  in  your  thesis  is 
that  the  third  parties  are  also  audited,  whereas,  no  one 
audits  DC AS .  And  also  the  third-party  auditors  are  all 
certified  through  the  European  [accreditation  process] . 
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All  of  the  auditors  that  these  folks  use  are  certified  by 
them.  So  you  are  coming  in  with  some  extremly  heavy 
weights  instead  of  someone  that's  very  well  versed  in  one 
topic.  The  benefit  of  that  is  that  these  folks  will  come 
in  and  they  do  an  in-depth  three-to-f ive-day  survey, 
[every  six  months]  much  like  you  do  with  the  QARs 
initially  or  every  three  to  five  years. 


The  main  complaint  concerning  Government 
auditors  appears  to  be  a  lack  of  training  and  an  adversarial 
attitude.  It  should  be  pointed  out  that  the  ISO  auditors  are 
paid  by  the  company  they  are  auditing  and  thus  probably  treat 
the  people  they  are  auditing  as  their  customers.  This  does 
not  imply  that  the  ISO  auditors  aren't  rigorous  in  their 
audits.  They  do  have  their  reputation  and  professional 
standards  to  maintain,  but  an  auditor  does  not  have  to  assume 
an  adversarial  attitude  to  be  effective. 

The  second  area  of  difference  noted  is 
directly  tied  to  the  level  of  training  and  education.  The 
ISO-registering  companies  are  sending  experienced  and  highly 
trained  professionals  to  conduct  these  audits.  These 
professionals  usually  have  technical  or  engineering 
backgrounds  and  frequently  have  extensive  experience  in  the 
industry  they  are  auditing.  This  level  of  professionalism 
makes  communication  with  the  audited  company  much  easier;  the 
company  being  audited  is  not  "training"  the  auditor. 

The  entire  question  of  how  to  implement  ISO 
9000  brings  with  it  opportunities  to  improve  the  way  DoD 
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administers  quality  assurance  oversight.  At  the  same  time, 
any  changes  implemented  in  this  field  bring  with  it  the  risk 
of  aggravating  the  adversarial  element  found  in  some  areas  of 
DoD  contracting  . 

4.  Question  Four. 

If  DoD  were  to  impose  ISO  9000  Standards,  including 
registration,  what  are  the  barriers  that  a  company  faces  in 
complying  with  ISO  9000? 

a .  Discussion 

The  cost  of  registration  is  the  obvious  starting 
point  when  discussing  the  impact  of  imposing  ISO  9000 
standards,  but  the  researcher  was  also  interested  in 
determining  if  there  were  other  barriers  created  when  ISO  9000 
is  contractually  imposed.  The  cost  to  become  ISO  9000 
registered  varies  dramatically,  based  on  several  factors 
including : 

•  what  registration  authority  is  selected  to  perform  the 
audits  and  registration; 

•  the  condition  of  the  quality  assurance  program  before  the 
initial  registration  audits  are  performed;  and 

•  the  size  and  -type  of  business  that  is  applying  for 
registration . 

b.  Analysis 

The  registrars  work  on  a  cost-per-hour  basis.  If 
a  company  is  prepared  before  the  registrar  makes  the  initial 
audit  and  review  of  the  company's  quality  program,  than  the 
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time  spent  by  the  registrar  is  greatly  reduced  and  cost  of 
registration  is  lowered.  There  are  numerous  consulting  firms 
that  provide  pre-registration  preparation  services  to  help 
reduce  the  time  and  expense  associated  with  the  registration 
process.  Thus,  it  becomes  difficult  to  determine  the  cost  of 
registration  for  any  given  company  or  industry.  A  firm  that 
has  a  solid  quality  assurance  program  in  place  will  greatly 
reduce  its  registration  expense  in  contrast  to  another  company 
of  comparable  size  and  industry  that  is  using  only  a  limited 
program  of  end-item  inspection.  The  interviewees  frequently 
mentioned  that  a  firm  operating  under  MIL-Q-9858  is  well  on 
the  way  to  ISO  9000  compliance  and  thus  faces  a  lower 
registration  cost. 

The  researcher  followed  up  the  comments  on  cost 
(given  registration  costs  were  a  problem)  with  the  question: 
are  there  any  other  barriers  that  ISO  imposes  that  would  keep 
a  company  from  completing  registration?  The  answers  in  this 
area  fell  into  the  following  four  different  categories. 

(1)  The  Scope  of  ISO  is  Too  Broad.  In  the  first 
category  are  those  comments  that  describe  ISO  9000  as  a  broad 
document  that  requires  a  great  deal  of  effort  on  the  part  of 
management  to  interpret  and  customize  in  order  to  implement 
it.  A  sample  of  these  is  paraphrased  below: 

•  This  ISO  document  is  a  short  document  that  has  very  little 
in  terms  of  how  to  do  anything.  What  it  does  tell  you  is 
what  you  must  do.  Design  is  a  good  example;  it  says  you 
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must  state  the  design  requirements,  you  must  implement  the 
design,  demonstrating  that  you  are  complying  with  the 
requirements,  you  must  verify  [and]  have  the  documentation 
to  show  you  have  satisfied  the  design  requirements  and  you 
must  verify  independently.  That's  all  it  says;  it  doesn't 
tell  you  how  to  do  it. 

•  ...  not  that  it  is  complicated,  but  I  see  companies 
struggling  with  it  only  because  the  standard  itself  is  not 
articulate  so  that  you  can  go  and  address  specific  areas; 
there  is  some  interpretation  involved. 

•  The  hardest  part  with  ISO  is  to  make  sure  that  you 
interpret  it  in  the  manner  that  it  needs  to  be 
interpreted. 

•  The  ISO  9000  in  my  mind  is  basically  a  blank  sheet  of 
paper.  It  is  a  tell-us-what-it-is-you  -do .  It  gives 
minimal  guidance  in  the  implementation  of  our  quality 
process . 


These  comments  seem  to  imply  that 
implementation  of  ISO  9000  Standards  requires  the  company 
management's  commitment  to  review  and  modify  its  quality 
system,  based  on  broad  guidelines  that  require  upper 
management  interpretation.  This  indicates  that  ISO  9000  is 
not  a  cookbook  approach  to  implementing  a  new  quality  system 
in  a  company.  It  requires  the  effort  and  time  of  the  entire 
organization  to  successfully  accomplish  ISO  registration. 

(2)  Differences  in  Philosophy .  The  second  common 
category  of  comments  flows  out  of  the  interpretation  issue 
raised  above.  Several  of  the  people  interviewed  clearly  drew 
a  distinction  between  the  philosophy  underlying  ISO  9000 
versus  the  normal  U.S.  approach  to  the  design  of  a  quality 
system.  The  interviewees  all  point  to  ISO  being  a  step  toward 
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continuous  improvement  and  Total  Quality  Management.  To  the 
extent  that  ISO  represents  a  fundamental  change  in  the  way 
companies  view  their  quality  assurance  efforts,  it  can  prove 
to  be  a  barrier  to  a  company's  willingness  to  adopt  ISO 
Standards.  Add  to  this  the  fact  that  a  lot  of  companies  are 
quite  satisfied  with  their  current  quality  assurance  systems 
and  the  imposition  of  ISO  Standards  could  well  eliminate 
potential  suppliers  from  obtaining  DoD  contract  awards.  A  few 
of  the  representative  comments  in  this  area  are  paraphrased 
below: 


•  ...  a  lot  of  companies  have  vertical  quality  control  and 

quality  assurance.  What  I  mean  by  that  is  the  technical 
product  control,  if  you  will,  within  the  actual 

manufacturing  process  and  maybe  even  the  design  and 
development  process,  but  the  ISO  9000  is  addressing  the 
whole  company.  And  this  is  what's  new  to  U.S.  businesses. 
It  goes  with  the  whole  issue  of  continuous  improvement  and 
Total  Quality  Management. 

•  Yes  [there  are  barriers  other  than  just  cost]  and  it's 
primarily  that  some  companies  have  quality  systems  in 
place  that  do  not  parallel  ISO  9000.  They  feel  their 
quality  systems  are  very  adequate  and  are  very  happy  with 
them,  and  they  are  producing  quality  products.  This  might 
be  especially  true  in  the  case  of  a  larger  company 
let's  say  Motorola  or  something  like  that  where  if  you 
have  to  go  in  and  make  changes  to  the  system  it's  [impact 
will  be  felt  throughout  the  organization] .  Whereas  in  a 
smaller  company  where  you  can  quickly  make  a  little 
change,  it  does  not  impact  you.  If  you  have  to  make  a 
massive  change  across  the  board  [in  a  large  company] ,  you 
have  to  communicate  it  to  all  people,  put  new  procedures 
in  place,  etc.  So  there  is  that  concern  among  those 
companies . 
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(3)  Organizational  Resistance  to  Change.  The 
third  category  of  comments  is  simply  the  age-old  problem  of  an 
organization's  resistance  to  change.  ISO  represents  change  to 
many  of  the  firms  implementing  it.  A  few  of  the 
representative  comments  along  these  lines  are  paraphrased: 

•  It  is  people's  resistance  to  change,  that  is  the  biggest 
one.  There  are  no  drawbacks  to  it,  other  than  the  fact 
that  it  does  cost  you  a  couple  of  bucks.  But  people  that 
have  been  doing  business  as  usual  for  so  long  are  so 
resistant  to  change;  that  is  probably  by  far  the  biggest 
hurdle  to  overcome. 

•  The  only  other  factor,  of  course,  is  that  it  is  new  and 
they  are  not  familiar  with  it,  and  they  would  have  to 
learn  it  and  learn  about  it. 

(4)  ISO  is  Difficult  to  Use  in  Some  Environments . 

This  fourth  category  came  from  people  who  were  responsible 

for  implementing  ISO  in  their  companies.  The  comments  dealt 
with  the  problem  of  implementing  ISO  in  an  environment  that  is 
not  purely  manufacturing.  Many  companies  in  the  high 
technology  fields  incorporate  some  creative  design  efforts 
within  their  manufacturing  systems  --  in  particular,  the 
computer  software  efforts  that  are  needed  to  support  many  of 
today's  high  tech  products.  These  people  found  implementing 
ISO,  where  there  is  no  clearly  defined  process,  to  be  a 
challenge.  Some  of  the  representative  comments  follow: 
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•  There  are  organizations  that  are  very  well  structured, 
typically  a  manufacturing  organization  and  somewhat  a 
service  organization,  or  even  a  hardware  engineering 
organization,  which  are  more  accustomed  to  rigor  and 
process.  If  you  try  to  implement  this  in  a  software  world 
where  there  is  more  freedom  of  product  expression,  most 
places  don't  have  that  level  of  rigor  in  their  software 
environment.  So  implementing  this  process  in  a  software 
world  will  require  a  fine  balance  of  a  process  which  is 
flexible  enough  to  allow  that  creativity  but  sufficiently 
defined  to  allow  a  defined  and  controlled  process  flow. 
And  that  is  a  fine  level  which  is,  I  think,  not  obvious  to 
implement . 

•  It  was  much  easier  and  quicker  [within  our  company]  for 
our  manufacturing/logistics  organizations  and  our  hardware 
engineering  organization  to  come  up  to  speed.  The 
software  side  [still]  has  a  long  haul. 

•  What  you  need  to  be  able  to  do  is  define  a  process  which 
gives  an  overview  or  the  right  templates  of  things  to 
address,  but  doesn't  make  everything  required  for  every 
implementation  because  of  the  different  size  of  the 
projects  [or]  scope  [or]  technology.  Basically  you  have 
to  give  a  framework  or  a  guideline  and  then  leave  it  up  to 
the  individual  implementation  of  what  makes  sense  for  that 
particular  project. 


This  last  set  of  cor.urents  raises  the  issue  that 
imposing  ISO  in  a  Research  and  Development  (R&D)  type  contract 
would  prove  to  be  extremely  difficult.  This  same  issue  arises 
when  judging  the  quality  of  R&D  or  software  under  MIL-STDS. 
The  current  solution  to  this  problem  is  to  impose  MIL-I-45208 
or  some  other  end-item  delivery  inspection.  ISO  9001  and  9002 
are  not  equivalent  to  MIL-I-45208  because  they  do  not  stress 
end-item  inspection.  The  underlying  theme  of  ISO  is  to  judge 
whether  the  quality  system  is  within  design  parameters  and  if 
there  is  room  for  improvement.  It  is  hard  to  make  these 
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judgments  when  the  process  is  not  well  defined  or  amenable  to 
quantifiable  measurement. 

5.  Question  Five 

If  you  were  given  the  opportunity  to  tell  DoD  what  it 
should  do  with  regards  to  ISO  9000,  what  would  you  say? 

a .  Discussion 

This  question  was  meant  to  elicit  general  comments 
and  provide  an  opportunity  to  benefit  from  the  collective 
experience  and  wisdom  of  this  unique  group  of  people. 

Jb .  Analysis 

The  question  yielded  a  wide  scope  of  answers  and 
what  is  paraphrased  below  are  those  comments  the  researcher 
felt  were  most  pertinent  to  a  DoD  Contracting  Officer. 


•  I  believe  that  unless  a  company  is  third-party  accredited 
and  there  is  some  formal  process  of  audit  and  review  of 
the  quality  system,  then  there  is  no  way  for  anybody  to 
tell  whether  that  company  or  group  is  really  performing 
and  adhering  to  that  standard.  You  are  required  to  do 
internal  audits,  but  there  is  no  way  anybody  can 
demonstrate  that  they  are  adhering  to  their  own  quality 
manual  or  quality  policy  statements  or  whatever,  unless 
there  is  some  form  of  quality  auditing  being  performed. 

•  I  believe  that  from  the  standpoint  of  us  competing,  and 
the  things  that  are  being  learned  these  days  on  how  to 
manage  and  control  products  and  processes,  I  think  you 
have  to  rethink  how  we've  historically  done  it  and  start 
utilizing  some  of  these  new  concepts.  If  we  don't,  then 
we  don't  compete  very  well  and  we  don't  do  things  very 
timely.  Our  company  has  shown  some  dramatic  changes.  Even 
though  we  struggle  in  different  areas,  it  is  an  ongoing 
kind  of  thing  and  just  by  focusing  on  process  management 
and  things  of  this  nature,  I  think  you  see  dramatic 
changes  in  the  way  companies  operate. 
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•  One  thing  that  is  obvious,  and  you  are  probably  well  aware 
of  it,  is  that  9001  is  not  nearly  enough.  There  are 
gaping  holes  in  those  standards  that  need  to  be  addressed 
to  have  a  truly  competitive  company.  For  example,  nowhere 
in  ISO  9001,  9002  or  9003  do  you  see  a  requirement  for  a 
total  quality  environment  in  a  company--one  that's  driven 
towards  quality  improvement  and  things  like  that.  You 
touch  on  it  in  the  corrective  actions  sections,  but  there 
is  nothing  in  there,  [like]  clause  3 . 6  of  9858  telling  you 
to  measure  quality  costs.  It  caused  a  lot  of 
consternation  in  industry,  but  it's  a  hell  of  a  good  idea. 


•  It  is  not  only  a  benefit  to  doing  a  better  job  of  managing 
our  business,  but  it's  also  a  vehicle  for  staying  in  the 
game  with  our  customers.  And  there  is  nothing  we  did  to 
achieve  compliance  that  didn't  make  good  business  sense. 

•  My  experience  in  complying  with  9858  goes  back  10  years. 

There  seemed  to  be  more  concentration  on  the  form  of  what 
the  document  requires,  as  opposed  to  the  substance  of  what 
the  requirement  of  the  document  was.  The  auditors  that 
came  in  spent  all  their  time  looking  at  whether  a  gauge 
was  out  of  calibration.  And  when  they  found  one, 
Hurrah!  That  was  a  big  victory  for  the  auditors  under 
9858.  That  philosophy  and  mentality  doesn't  seem  to 

persist  with  ISO. 

•  We  are  just  afraid  that  ISO  9000  in  the  next  five  years  is 
going  to  lower  its  standards.  We  don't  want  [ISO]  to 
lower  their  standards.  But  it  is  so  backed-up  like  you 
wouldn't  believe,  with  ISO  requests.  I  just  hope  they 
don't  lower  their  standards  because  of  the  workload. 

•  It  is  one  of  those  hard  things  to  quantify,  but  I  can 

judge  quality,  and  I  can  figure  out  the  cost  of  quality 
here.  We  recently  acquired  another  company  and  my 

credits,  as  far  as  customer  returns,  have  not  changed. 
Yet  I  have  doubled  my  line  items  from  this  building  alone. 
This  is  absorbing  inventory  from  three  major  warehouses 
and  our  quality  has  not  suffered.  Not  to  mention,  we 
recently  hired  30  new  people  and  the  training  that  we  had 
put  in  place,  has  been  working  out  fantastically.  I  have 
30  [new]  people  here  who  I  never  felt  that  training  curve 
as  far  as  our  credits  [customer  returns]  to  line  items  go. 
As  much  as  getting  the  ISO  9000  certification  did  for  us, 
from  a  sales  standpoint,  it  did  as  well  for  us  from  a 
quality  standpoint. 

•  [With]  ISO  I  have  a  concern  that  it  might  get  locked  in 
concrete.  That  could  be  a  problem,  along  with 
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interpretation  of  it  --  the  same  interpretation  problems 
you  get  in  MIL-Q-9858,  even  with  the  handbook  H50 
available  [are  inherent  in  ISO]  . 


•  It  is  a  very  positive  step  [to  go  to  ISO  9000]  .  However, 
there  is  a  risk  that  if  you  focus  too  much  on  process, 
then  you  lose  some  of  your  product  metrics  which  is  the 
traditional  QA  approach.  My  fear  there  is  that  process 
focus  is  good  long-term  business  effort,  but  if  it  is 
focused  just  on  process  without  tying  to  some  direct 
product  results,  you  can  easily  miss  the  mark  and  waste 
your  energy. 

These  general  comments  are  intended  to  highlight 
potential  future  problems  for  Contracting  Officers.  Each 
observation  sheds  valuable  light  on  such  situations.  And 
since  these  observations  come  from  experienced  ISO  9000 
quality  assurance  managers,  they  are  worth  noting. 

C.  SUMMARY 

The  telephone  surveys  revealed  an  enthusiastic  and  growing 
support  for  ISO  9000  among  professional  quality  assurance 
managers.  The  current  number  of  U.S.  ISO-registered  companies 
is  225.  As  this  initial  base  of  companies  increases,  the 
application  of  ISO  9000  Standards  in  DoD  contracts  will 
undoubtedly  grow  and  will  replace  MIL-Q-9858  and  MIL-I-45208. 
It  is  incumbent  on  a  Contracting  Officer  to  stay  abreast  of 
this  development  and  realize  the  potential  and  actual  impact 
of  making  this  change  in  quality  standards. 

Chapter  V  summarizes  the  data  discussed  and  analyzed  in 
this  chapter,  draws  conclusions,  supports  recommendations  for 
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implementation  of  ISO  in  DoD  contracts,  and  suggests  areas  for 
additional  research. 
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V.  CONCLUSIONS,  RECOMMENDATIONS  AND  AREAS  FOR  ADDITIONAL 

RESEARCH 

A.  INTRODUCTION 

This  study  has  attempted  to  assess  the  movement  toward 
adoption  of  ISO  9000  Standards  in  terms  of  DoD  contracting. 
In  order  to  make  that  assessment.  Chapter  II  discussed  the  new 
international  ISO  9000  Standards  and  their  equivalent  MIL- 
STDS .  This  discussion  was  followed  by  a  detailed  comparison 
of  the  two  sets  of  standards  in  Chapter  III.  Once  this  basic 
understanding  of  what  ISO  is,  versus  MIL-STDS,  the  researcher 
interviewed  current  users  of  ISO  9000  in  order  to  solicit 
their  input  regarding  the  use  of  ISO  Standards  in  Government 
contracting . 

Based  on  both  the  survey  performed  and  the  comparison  of 
ISO  tc  MIL-STDS,  the  researcher  has  developed  the  following 
conclusions,  recommendations,  answers  to  research  questions 
and  areas  for  additional  research. 

B.  CC  ELUSIONS 

On  3  of  the  purposes  of  this  study  was  to  provide  an 
assessment  of  the  current  movement  within  DoD  and  industry  as 
to  the  adoption  of  ISO  9000  Standards.  Regarding  that 
assessment,  the  first  conclusion  of  this  study  is: 
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1.  The  DoD  policy  toward  ISO  9000  is  not  currently  a 
factor  in  the  decision  of  companies  to  become  ISO  9000 
registered. 

The  responses  to  the  second  question  of  the  telephone 
survey  clearly  indicate  that  DoD  quality  assurance  policy  was 
not  a  factor  in  the  decision  to  become  ISO  9000  registered  for 
those  companies  that  have  completed  ISO  registration.  It 
should  be  noted  that  the  companies  contacted  represent  the 
leaders  in  the  movement  to  ISO  9000  Standards.  The  motivation 
of  these  leaders  is  based  on  their  marketing  strategies  for 
international  sales.  Each  of  these  companies  intends  to 
comply  with  ISO  9000  in  order  to  remove  a  barrier  to  their 
participation  in  international  markets,  and  in  particular,  the 
European  markets. 

In  the  event  that  DoD  makes  ISO  9000  compliance  a 
contractual  requirement,  this  conclusion  would  undoubtedly 
change  radically.  It  should  be  remembered  that  this  study  was 
conducted  during  a  time  when  ISO  9000  is  still  in  its  infancy 
in  the  U.S.  There  were  only  225  companies  that  were  ISO  9000 
registered  at  the  time  the  telephone  survey  was  conducted. 

The  conclusion  that  current  and/or  contemplated  DoD 
policy  is  not  a  factor  in  the  decision  to  be  ISO  registered, 
does  not  preclude  such  policies  from  becoming  key  factors  in 
ISO  registration,  once  it  becomes  widely  known  that  DoD  is 
considering  using  ISO  standards  in  place  of  applicable  MIL- 
STDS . 
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The  second  conclusion  of  this  study  is: 

2.  The  adoption  of  ISO  9001  in  place  of  MIL-Q-9858A  will 
have  no  major  cost  impact  for  those  companies  currently 
operating  under  MIL-Q-9858A. 

This  conclusion  is  drawn  from  the  comparison  of  ISO 
9001  to  MIL-Q-9858A,  presented  in  chapter  III,  and  from 
numerous  comments  made  by  people  during  the  telephone  survey. 
Several  contractors  remarked  incidentally  that  they  saw  no 
difference  between  ISO  9001  and  MIL-Q-9858A.  A  number  also 
stated  that  the  system  they  had  developed  for  compliance  with 
MIL-Q-9858A  covered  the  majority  of  the  compliance 
requirements  contained  in  ISO  9001.  The  OSD(P&L)  comparison 
cited  in  Chapter  III  also  concluded  that  ISO  9001  and  MIL-Q- 
9858A  are  fundamentally  compatible  as  to  scope  and  cost 
impact . 

The  significance  of  this  conclusion  to  Contracting 
Officers  will  become  apparent  in  future  meetings  where 
contractors  will  cite  ISO  9001  requirements  in  a  contract  as 
the  basis  of  increased  cost  to  fulfill  the  contract.  Any 
assertion  of  added  cost  due  to  the  use  of  ISO  9001,  from  a 
contractor  who  is  currently  conforming  to  MIL-Q-9858A,  should 
be  seriously  challenged.  To  the  extent  changes  to  quality 
assurance  procedures  are  required  to  conform  to  ISO  9001,  such 
changes  should  be  relatively  minor  in  scope  and  cost  impact. 
The  philosophy  of  ISO  9001  is  to  ensure  that  the  process 
remains  under  control,  and  emphasizes  feedback  that  leads  to 
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continuous  improvement  of  the  process.  In  addition,  any  costs 
incurred  in  changing  quality  assurance  procedures  should  be 
part  of  an  indirect  overhead  expense  pool.  These  costs  should 
be  borne  across  all  segments  of  a  firm's  business. 

If  a  contractor  were  to  cite  the  direct  costs  of 
becoming  ISO  9000  registered  as  a  basis  for  added  contract 
costs,  the  Contracting  Officer  has  two  responses.  First, 
registration  is  not  currently  a  DoD  requirement.  Second,  if 
DoD  were  to  make  it  a  requirement,  the  company  would  realize 
some  benefits  in  the  registration  process  that  offset  any 
direct  costs  incurred.  These  benefits  were  mentioned  by 
several  of  the  persons  interviewed  during  discussions 
concerning  survey  question  four.  The  cost  of  registration  was 
always  cited  as  a  barrier  to  adopting  ISO  9000,  but  several 
contractors  were  quick  to  point  out  that  the  registration 
process  helped  identify  weaknesses  in  their  current  quality 
assurance  systems.  These  weaknesses  had  to  be  corrected  prior 
to  achieving  registration;  thus  the  registration  process  acted 
as  a  catalyst  for  improvements  that  yield  future  benefits. 
This  correction  process  was  considered  by  several  quality 
assurance  managers  to  be  a  significant  source  of  improvement 
that  will  bear  tangible  savings  in  the  near  future.  An 
independent  third  party,  coming  in  to  review  a  company's 
entire  quality  assurance  system,  is  bound  to  generate  new 
ideas  and  points  of  view  that  incumbents  of  a  system  have 
failed  to  recognize.  The  quotes  cited  in  Chapter  IV 
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concerning  the  perceived  quality  of  ISO  9000  auditors  adds 
weight  to  this  line  of  reasoning.  The  perception  is  that  ISO 
auditors  are  professionals  with  significant  education  and 
experience.  The  respondents  all  conveyed  the  feeling  that  ISO 
registration  is  a  meaningful  process.  The  companies  contacted 
had  paid  significant  sums  of  money  to  achieve  registration. 
It  can  be  reasonably  concluded  that  those  expenditures  were 
subjected  to  the  normal  review  and  justification  criteria 
inherent  in  organizations  that  are  upgrading  the  way  they  do 
business.  The  initial  impetus  may  have  come  from  marketing, 
but  the  respondents  all  made  a  strong  case  for  the  benefits 
derived  by  improvements  in  their  quality  assurance 
departments,  brought  about  by  ISO  registration. 

Regardless  of  whether  a  contractor  cites  direct  or 
indirect  costs,  the  conclusion  of  this  researcher  is  that 
using  ISO  9001  in  place  of  MIL-Q-9858A  is  not  reasonable 
justification  for  such  additional  costs. 

The  third  conclusion  of  this  study  is: 

3.  The  adoption  of  ISO  9002  in  place  of  MIL-I-45208  will 
impact  the  cost  for  those  companies  currently  operating  under 
MIL-I-45208. 

The  comparison  presented  in  Chapter  III  between  ISO 
9002  and  MIL-I-45208  clearly  shows  that  ISO  9002  is 
structurally  and  philosophically  a  different  standard  than 
MIL-I-45208.  ISO  9002  takes  ISO  9001  and  deletes  those 
paragraphs  that  apply  to  quality  control  of  the  design 
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process.  ISO  9002  still  has  a  continuous  improvement  element 
and  stresses  control  of  the  process.  MIL-I-45208  was 
summarized  by  many  respondents  as  an  end-item  acceptance 
inspection.  It  does  not  attempt  to  integrate  the  quality 
assurance  function  into  the  entire  process;  it  simply  strives 
to  detect  defective  material  after  it  has  been  produced.  The 
implementation  of  ISO  9002  in  a  company  operating  under  MIL-I- 
45208  would  require  the  formalization  of  procedures,  and 
documentation  of  the  entire  process,  leading  up  to  end-item 
inspection.  This  increased  documentation  and  control  results 
in  added  costs,  both  immediate  costs  to  establish  the  systems, 
and  the  continuing  costs  of  maintenance.  The  trade-off  to 
these  costs  should  be  the  realization  of  lower  scrap  and 
rework  charges  that  arise  under  an  end-item  inspection  system. 
Regardless  of  possible  offsetting  benefits,  a  review  of  the 
differences  between  ISO  9002  and  MIL-I-45208  shows  that  there 
is  added  cost  in  changing  from  MIL-I-45208  to  ISO  9002. 

The  Contracting  Officer  who  uses  ISO  9002  as  a 
contractual  requirement  must  be  prepared  to  deal  with  the 
added  costs  that  will  be  incurred  by  this  change  to  a 
contractor's  quality  assurance  system. 

The  fourth  conclusion  of  this  study  is: 

4.  The  implementation  of  ISO  9000  Standards  will  still 
require  the  same  type  of  amplifying  policies  and  procedures 
that  are  currently  used  with  MIL-STDS. 
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There  is  a  widespread  misconception  among  even 
knowledgeable  quality  assurance  managers  that  MIL-Q-9858A  and 
MIL-I-45208  are  huge  detailed  documents  more  than  an  inch 
thick.  In  reality,  both  these  MIL-STDS  are  brief  concise 
documents  that  reference  other  standards  and  handbooks, 
resulting  in  layer  upon  layer  of  documents  that  detail  the 
implementation  of  a  quality  assurance  system  commonly  known  as 
"MIL-Q."  These  layers  of  added  definition  and  direction 
evolve  and  change  in  response  to  the  problems  experienced. 
The  ISO  standards,  on  the  other  hand,  were  first  issued  in 
1987  and  there  has  not  been  enough  time  for  them  to  generate 
this  layering.  As  discussed  in  Chapter  II,  the  TC176 
committee  is  currently  working  on  changes  and  amplification  of 
the  basic  ISO  9000  series  of  standards,  and  the  layering 
effect  will  soon  become  apparent.  The  counterpart  to  many  of 
these  TC176  initiatives  can  be  found  in  the  handbooks  and 
references  associated  with  MIL-Q-9858A. 

The  fifth  conclusion  of  this  study  is: 

5.  A  requirement  for  ISO  9000  registration  in  a  DoD 
contract  would  create  resistance  to  that  requirement  from  some 
DoD  contractors. 

As  pointed  out  by  one  of  the  respondents  to  the 
telephone  interview,  some  companies  are  receiving  the 
information  and  level  of  quality  they  want  from  their  current 
quality  assurance  system.  These  companies  will  not  willingly 
bear  the  costs  required  to  achieve  ISO  9000  registration  until 
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the  benefits  of  registration  are  better  defined  and 
quantified . 

The  sixth  conclusion  of  this  study  is: 

6.  The  use  of  ISO  9000  requires  a  different  philosophy 
regarding  quality  assurance  than  that  required  to  operate 
under  MIL-STDS. 

The  commitment  of  a  company's  top  management  to  the 
major  tenets  of  ISO  9000  is  a  requirement  that  must  be 
satisfied  with  a  published  and  actively-supported  policy. 
Several  of  the  respondents  made  it  clear  that  prior  to 
granting  registration,  ISO  auditors  look  closely  at  the 
corporate  management  attitudes  toward  quality  assurance.  The 
ISO  auditors  were  al$u  described  as  "consultants"  by  one 
respondent,  and  several  mentioned  the  impression  that  the  ISO 
auditors  were  v'orking  with  them  to  improve  the  overall 
process,  rather  than  simply  inspecting  it  for  compliance. 
This  fostering  the  attitude  of  continuous  improvement  by  ISO 
is  one  of  the  key  differences  between  ISO  9000  and  MIL-STDS 
and  it  cannot  be  overemphasized.  ISO  standards  require 
feedback  from  the  user  and  continual  monitoring  of  the 
production  process  to  ensure  it  is  within  tolerance  levels. 

If  ISO  standards  are  to  be  used  in  place  of  MIL-STDS, 
this  same  change  in  philosophy  from  Government  auditors  would 
be  necessary  to  ensure  that  maximum  effectiveness  is  gained 
from  the  switch  to  ISO  standards.  Government  auditors  would 
have  to  focus  their  attention  on  aiding  the  contractor  in 
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improving  his  process  by  identifying  weak  or  out-of -tolerance 
areas.  This  approach  to  quality  assurance  is  new  to  the 
Government,  and  has  taken  form  in  a  program  labeled  In-plant 
Quality  Evaluation  (IQUE) .  IQUE  embraces  the  idea  of  the 
Government  Quality  Assurance  Representative  (QAR)  becoming  a 
member  of  the  contractor's  team  to  ensure  that  a  quality 
product  is  delivered  to  the  Government.  The  implementation  of 
ISO  9000  Standards  would  reinforce  this  philosophy  and 
strengthen  this  new  program. 

C .  RECOMMENDATIONS 

The  secondary  objective  of  this  study  was  to  look  at  ISO 
9000  and  see  how  it  could  be  implemented  within  the  DoD 
contracting  arena.  It  is  assumed  that  DoD  will  adopt  ISO  9000 
Standards  and  it  is  the  implementation  of  those  standards  that 
generates  the  following  six  recommendations. 

The  first  recommendation  of  this  study  is: 

1.  DoD  should  define  its  policy  of  replacing  MIL-Q-9858 
with  ISO  9001,  and  begin  integrating  it  into  DoD  contracts. 
This  integration  should  be  accompanied  by  an  education  program 
for  Contracting  Officers  covering  the  costs  and  benefits  of 
ISO  9001,  as  compared  to  MIL-Q-9858A. 

The  quality  assurance  side  of  DoD,  as  represented  by 
OSD  ( P&L )  ,  has  decided  that  ISO  9001  is  an  improvement  over 
MIL-Q-9858A  and  it  wants  to  impose  these  standards  on  DoD 
contractors.  This  decision  is  based  on  several  factors,  but 
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a  most  important  element  to  bear  in  mind  is  that  imposing  ISO 
9001  on  a  contractor  already  using  MIL-Q-9858A  should  not 
result  in  significant  added  cost.  This  change  to  ISO  9001 
promises  to  hold  benefits  in  better  quality  products,  along 
with  reducing  the  adversarial  atmosphere  that  is  prevalent  in 
some  industries  between  QARs  and  contractors.  By  accepting 
ISO  9001,  contractors  are  aided  in  achieving  standardization 
of  a  key  element  in  their  business,  thus  reducing  the 
redundancy  that  contractors  experience  when  trying  to  maintain 
two  systems  --  one  for  DoD  and  one  for  commercial  markets. 

To  make  the  transition  from  a  purely  technical 
decision  to  implementing  ISO  9001  in  a  contract  will  require 
that  Contracting  Officers  be  given  a  basic  understanding  of 
the  impact  this  change  will  cause.  If  a  contractor  approaches 
a  Contracting  Officer  about  ISO  9001,  it  is  incumbent  on  the 
Contracting  Officer  to  provide  a  response  based  on  at  least  a 
fundamental  knowledge  of  the  new  system. 

The  education/training  is  not  limited  to  Contracting 
Officers.  DoD  quality  assurance  personnel,  especially  QARs, 
need  to  be  retrained  in  the  manner  in  which  they  approach  the 
subject  of  quality  assurance.  As  pointed  out  in  conclusion 
seven,  imposition  of  ISO  9000  on  a  contractor  requires 
acceptance  of  a  new  philosophy,  which  calls  for  better 
communication  and  cooperation  between  quality  auditors  and  the 
production  personnel  involved. 
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The  recommendation  to  implement  ISO  9001  in  contracts 
leads  directly  to  the  second  recommendation: 

2.  DoD  should  train  and  educate  their  QARs  in  ISO  9000, 
and  adopt  and  apply  its  philosophy  to  quality  audits. 

The  support  of  IQUE,  as  mentioned  in  conclusion  seven, 
is  but  one  example  of  the  benefit  DoD  could  realize  by 
actively  supporting  the  retraining  of  QARs  to  ISO  9000 
Standards.  DoD  has  recognized  that  its  quality  operations  in 
the  past  have  not  achieved  the  desired  level  of  cooperation 
and  support  from  contractors.  ISO  9000  gives  DoD  an 
opportunity  to  improve  both  the  contractor's  and  DoD's  quality 
procedures  by  getting  both  sides  to  agree  to  use  the  same  set 
of  standards.  Hopefully,  both  will  learn  to  use  ISO  9000  in 
a  way  that  improves  the  production  process,  thereby  improving 
quality . 

The  third  recommendation  of  this  study  is  meant  to 
support  an  atmosphere  of  cooperation  that  ISO  9000  makes 
possible : 

3.  The  idea  of  using  ISO  9000  registrars  as  arbitrators 
in  disputes  between  DoD  and  contractors  should  r  t  be  adopted. 

This  idea  of  finding  an  alternative  disputes 
resolut-irn  (ADR)  method  for  settling  quality  issues  before 
they  become  legal  issues,  has  appeal  at  first  glance.  The 
problem  with  using  a  registrar  as  an  arbitrator  lies  in  the 
risk  of  reinforcing  the  adversarial  relationship  between  DoD 
and  contractors.  ISO  standards  support  a  goal  of  obtaining 
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feedback  from  customers  and  using  it  to  improve  the  process. 
The  use  of  ADR  instead  of  open  communication  regarding 
problems  puts  DoD  back  into  an  adversarial  role  with 
contractors . 

The  introduction  of  registrars  into  dispute 
resolutions  also  means  that  a  company  is  now  faced  with  a  new 
antagonistic  relationship,  forced  by  DoD  regulation.  This  new 
relationship  is  between  the  company  and  the  registrar.  The 
idea  behind  ISO  registration  is  to  provide  an  experienced  and 
helpful  review  of  a  quality  assurance  program,  not  to  act  as 
a  third-party  buffer  between  a  company  and  its  customers. 

During  the  telephone  interviews,  it  became  clear  that 
the  quality  managers  and  directors  contacted  felt  any  disputes 
regarding  quality  could  be  settled  at  a  lower  level,  rather 
than  by  an  outside  third  party.  These  same  managers  asserted 
that  if  DoD  QARs  were  properly  trained  in  ISO  standards,  the 
poor  communication  between  them  and  contractor  personnel  would 
be  reduced,  since  both  sides  should  now  have  the  same  set  of 
policies  and  procedures  to  use  in  resolving  disputes. 

This  use  of  ISO  9000  to  reduce  friction  is  taken  one 
step  further  in  recommendation  four  by  reducing  the  workload 
experienced  by  both  the  contractor  and  DoD  personnel.  The 
fourth  recommendation  is: 

4.  Successful  completion  of  ISO  registration  should  be 
equivalent  to  a  satisfactory  pre-award  survey  of  a 
contractor's  quality  assurance  system. 
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The  ISO  auditors  use  almost  exactly  the  same  procedure 
used  during  pre-award  surveys  by  QARs  and  Administrative 
Contracting  Officers.  They  review  the  policies  and  procedures 
of  a  company  for  adequacy,  and  then  audit  the  quality 
assurance  process  to  see  if  it  is  conforming  to  stated  policy 
and  procedures.  DoD  would  better  utilize  its  scarce  resources 
by  eliminating  pre-award  quality  surveys,  thereby  reducing  the 
workload,  while  gaining  the  benefit  of  experienced  teams  of 
highly  trained  professional  auditors,  doing  the  work  of  pre¬ 
award  quality  surveys  at  no  cost  to  DoD.  The  ISO  auditors 
registering  companies  to  ISO  9000  Standards  must  themselves  be 
audited  and  certified  as  acceptable  registrars.  This  auditing 
and  certification  of  registrars  is  currently  based  on  European 
governmental  oversight  through  the  qualification  process  for 
a  company  to  become  certified  as  a  registrar. 

The  fifth  recommendation  of  this  study  is: 

5.  Impose  ISO  9000  only  in  those  contracts  in  which  it 
makes  sense. 

ISO  9000  standards  are  designed  for,  and  heavily 
weighted  toward,  the  creation  of  a  quality  assurance  system 
for  production  processes.  The  standards  are  written  in  such 
a  way  that  implementation  is  much  easier  where  the  production 
process  of  the  end  item  is  well  defined.  As  stated  in  the 
telephone  surveys,  and  supported  by  a  review  of  the  standards, 
ISO  9000  is  not  easily  applied  to  situations  that  require 
creative  production  processes. 
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The  comparison  of  ISO  9002  to  MIL-I-45208  also 
demonstrates  that  ISO  standards  may  not  be  used  as  exact 
replacements  for  MIL-STDS.  Imposing  ISO  9002  on  a  contractor 
not  used  to  anything  but  end-item  inspection  will  result  in 
added  cost,  as  explained  in  conclusion  three.  If  ISO  9002  is 
to  be  a  contractual  requirement,  the  cost  impact  needs  to  be 
acknowledged  and  addressed. 

The  sixth  recommendation  of  this  study  is: 

6.  Implementation  of  ISO  standards  in  DoD  contracts  must 
be  done  over  a  reasonable  period  of  time. 

Any  move  toward  contractual  requirements  to  use  ISO 
9000  Standards  will  take  an  extended  period  of  time  to  phase 
in.  There  were  only  225  companies  registered  to  ISO  9000  at 
the  time  the  telephone  survey  was  made,  and  thus  there  are  a 
great  number  of  companies  that  have  to  learn  about  ISO  9000, 
and  expend  resources  to  implement  it,  before  it  could  be  taken 
as  a  DoD-wide  requirement. 

The  phase-in  period  must  be  further  extended  if 
recommendation  four,  which  calls  for  the  use  of  registration 
as  a  means  to  reduce  pre-award  quality  surveys,  is  fully 
implemented.  Since  most  companies  are  not  ISO  registered, 
registration  costs  could  conceivably  create  a  barrier  to 
competition;  any  use  of  registration  should  be  as  an 
alternative  to  existing  procedures,  to  allow  sufficient  time 
for  DoD  contractors  to  analyze  and  decide  if  registration 
costs  are  worth  the  benefits.  It  is  not  reasonable  to  demand 
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ISO  registration  when  some  companies  may  not  feel  the  need  to 
change  from  their  existing  system.  In  addition,  registrars 
are  experiencing  a  demand  for  their  services  to  register 
companies,  that  exceeds  their  capacity  to  supply. 

D.  ANSWERS  TO  RESEARCH  QUESTIONS 

The  primary  research  question  of  this  study  is:  Should 
ISO  9000  be  implemented  within  DoD  and  the  Defense  Industrial 
Base,  and  when  implemented,  how  will  it  affect  DoD 
contracting? 

The  answer  to  this  question  is  contained  in  the  previously 
discussed  conclusions  and  recommendations,  but  can  be  briefly 
summarized  as  follows: 

1.  Put  ISO  9001  in  place  of  MIL-Q-9858A  in  future 
contracts . 

The  cost  impact  of  complying  with  ISO  9001  instead  of 
MIL-Q-9858A  for  most  contractors  is  negligible.  At  the  same 
time,  the  benefits  of  adopting  an  improvement -oriented  quality 
assurance  attitude  are  great.  The  use  of  a  standard  quality 
assurance  policy,  aimed  at  continuous  improvement  for  both 
commercial  and  DoD  work,  will  result  in  economic  savings  to 
the  contractor  and  better  quality  products  for  DoD. 

2.  Do  not  make  it  a  contractual  requirement  that  a 
contractor  be  registered  to  ISO  standards. 

There  were  only  225  companies  in  the  U.S.  that  were 
registered  at  the  time  this  study  and  survey  was  conducted. 
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This  small  number  of  ISO  registered  companies  means  that  a 
registration  requirement  would  create  barriers  to  competition. 

3.  The  primary  impact  of  ISO  9000  will  be  to  change  the 
philosophy  of  DoD  quality  assurance  programs  from  one  of 
inspection  to  continuous  improvement  and  prevention  of  poor 
quality  products. 

This  change  in  philosophy  will  have  both  cost  impact 
and  benefits.  The  Contracting  Officer  needs  to  make  a 
determination  that  the  benefits  outweigh  the  cost  before 
advocating  and  supporting  the  use  of  ISO  9000.  This 
determination  needs  to  be  made  on  a  contract-by-contract 
basis.  Answers  to  secondary  research  questions  can  be  found 
throughout  this  study;  they  are  briefly  summarized  below: 

1.  What  is  ISO  9000? 

ISO  9000  is  a  set  of  quality  assurance  standards 
created  to  provide  uniformity  among  the  European  Community, 
starting  in  1992.  These  standards  stress  process  control  and 
are  a  step  toward  continuous  improvement.  Chapter  II,  Part  B, 
presents  a  detailed  explanation  of  each  of  the  ISO  9000 
Standards . 

2.  What  are  the  similarities  and  differences  between  ISO 
standards  and  MIL -STDS? 

Both  sets  of  standards  are  process-control  related, 
but  ISO  has  a  fundamentally  different  philosophy  that  stresses 
improvement  and  active  involvement  of  production  with  quality 
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control  issues.  The  differences  between  the  two  sets  of 
standards  are  set  forth  in  Chapter  III  of  this  study. 

3.  What  is  the  policy  of  DoD  with  regards  to  ISO  9000? 

Its  stated  policy  is  to  move  toward  implementation  of 
ISO  standards  in  DoD  contracts,  but  not  to  require 

registration.  An  in-depth  review  of  DoD  policy  is  presented 
in  Chapter  II,  Part  D,  subpart  2. 

4.  What  are  the  anticipated  benefits  from  implementing 
ISO  standards? 

An  improvement  in  the  quality  of  products  delivered 
to  DoD  is  anticipated  by  changing  the  focus  of  quality 

assurance  from  end-item  inspection  to  one  of  process  control 
and  feedback.  The  ISO  standards  provide  DoD  with  the 
opportunity  to  assist  contractors  by  implementing  one  standard 
that  can  be  used  in  both  commercial  and  defense  work,  with 
regards  to  quality  assurance;  it  provides  a  step  toward 

improvement  of  quality  assurance  systems.  These  benefits  are 
evidenced  in  Chapter  IV  by  the  discussion  and  presentation  of 
telephone  interviews,  conducted  with  quality  managers  of 
companies  that  have  implemented  ISO  9000. 

5 .  What  steps  need  to  be  taken  to  implement  ISO 

standards? 

Steps  that  should  be  taken  in  this  area  involve  the 
following  actions: 

a.  Educate  and  train  personnel  in  ISO  9000  standards. 
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This  education  process  will  be  aimed  at  primarily 
DoD  and  contractor  QARs;  but  contracting  and  program 
management  personnel  will  also  need  to  have  a  general 
understanding  of  these  new  standards. 

b.  Invoke  ISO  90C1  in  place  of  MIL-Q-9858A  in  DoD 
contracts,  and  provide  the  option  of  using  ISO  9002  in  place 
of  MIL-I-45208 . 

The  time  has  come  to  move  ISO  9000  out  the  realm 
of  quality  assurance  specialists  and  implement  it  in  DoD 
contracts.  Comparisons  and  analyses  discussed  in  Chapters  II 
and  III  clearly  demonstrate  that  ISO  9000  is  superior  to  the 
current  DoD  practices  regarding  quality  assurance. 

c.  Assist  communication  with  contractors  as  to  ISO 
9000  and  inform  the  DoD  industrial  base  concerning  its 
existence. 

There  are  still  large  segments  of  DoD  and  industry 
which  are  not  aware  of  the  ISO  movement  and  what  it  means.  In 
order  to  invoke  fully  ISO  9000,  it  becomes  necessary  to 
convert  the  DoD  industrial  base  to  its  use. 

The  entire  idea  of  ISO  9000  is  so  new  that  it 
easily  gives  rise  to  additional  areas  for  further  research. 
A  few  of  these  areas  are  delineated  below: 
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E.  AREAS  FOR  ADDITIONAL  RESEARCH 


Recommended  topics  for  further  research  include: 

1.  A  cost-benefit  analysis  for  implementing  ISO  9001  in 
place  of  MIL-Q-9858A,  and  a  separate  analysis  of  ISO  9002 
rather  than  MIL-I-45208. 

2.  Study  of  education  requirements  for  DoD  personnel 
regarding  ISO,  and  determination  of  the  best  means  of 
satisfying  those  requirements. 

3.  Costs  and  benefits  companies  can  expect  from  ISO 
registration . 

4.  Case  study  of  a  company  that  has  implemented  ISO 
standards  to  determine  the  actual  benefits  of  using  these 
standards . 
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APPENDIX  A 


LIST  OF  TELEPHONE  INTERVIEWS 

1.  Aeroquip  Corporation  -  Aerospace  Division,  Jackson,  MI. 

2 .  American  Flange  &  Manufacturing  Company  Incorporated,  Carol 
Stream,  IL. 

3.  Arrow  Electronics  Incorporated,  Brookhaven,  NY. 

4.  Beloit  Corporation  -  Paper  Machine  Division,  Beloit,  WI . 

5.  Brand-Rex  Company,  Willimantic,  CT. 

6.  Copolymer  Rubber  &  Chemical  Company,  Baton  Rouge,  LA. 

7.  Dentsply  International  Incorporated,  Long  Island  City  NY 

8.  Digital  Equipment  Corporation,  Greenville,  SC. 

9.  DSC  Communications  Corporation,  Plano,  TX. 

10.  Fischer  &  Porter,  Warminister  PA 

11.  Flo-Bend  Incorporated,  Tulsa  OK 

.  12.  FMC  Corporation  -  Fluid  Control  Division,  Stephenville  TX 

13.  The  Foxboro  Company,  E.  Bridgewater  MA 

14.  General  Electric  Aircraft  Engines,  Lynn  MA 

15.  Harnischf eger  Corporation  -  Industrial  and  Electric  Products 
Group,  Milwaukee  WI 

16.  Lutron  Electronics,  Coopersburg  PA 

17.  Magnetrol  International  Incorportated,  Downers  Grove  IL 

18.  Prime  Computer  Incorporated,  Bedford  MA 

19.  RAM  Electronics,  Long  Beach  CA 

20.  Reliable  Automatic  Sprinkler  Company,  Mount  Vernon  NY 

21.  Rohrback  Cosasco  Systems  Incorporated,  Santa  Fe  Springs  CA 

22.  Rosemount  Analytical,  Orrville  OH 
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23.  STC  Submarine  Systems  Incorportated,  Portland  OR 

24.  Valmet -Appleton  Incorporated,  Appleton  WI 

25.  Yarway  Corporation,  Blue  Bell  PA 
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APPENDIX  B 


COMPARISON  OF  MIL-Q-9858A  TO  ISO  9001 


MIL-Q-9858A 

ISO  9001 

No  equivalency 

4  Quality  system  requirements 

4.1  Mangement  resonsibility 

4.1.1.  Quality  policy 

The  supplier's  management  shall 
define  and  document  its  policy 
and  objectives  for,  and 
commitment  to,  quality.  The 
supplier  shall  ensure  that  this 
policy  is  understood, 
implemented  and  maintained  at 
all  levels  in  the  organization. 

4.1.2  Organization 

4. 1.2.1  Responsibility  and 
authority 

The  responsibility,  authority 
and  the  interrelation  of  all 
personnel  who  manage,  perform 
and  verify  work  affecting 
quality  shall  be  defined; 
particularly  for  personnel  who 
need  the  organizational  freedom 
and  authority  to: 

a)  initiate  action  to  prevent 
the  occurrence  of  product 
nonconformity; 

b)  identify  and  record  any 
product  quality  problems; 

c)  initiate,  recommend  or 
provide  solutions  through 
designated  channels; 

d)  verify  the  implementation 
of  solutions; 
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"  ’  to  identify  and  evaluate 

problems  and  to 
e,  recommend  or  provide 
ns .  Management  regularly 
eview  the  status  and 
'adequacy  of  the  quality  program. 
|The  term  "quality  program 
[requirements"  as  used  herein 
identifies  the  collective 
requirements  of  this 
specification.  It  does  not  mean 
that  the  fufillment  of  the 
requirements  of  this 
specification  is  the 
responsibility  of  any  single 
contractor's  organization, 
function  or  person. 


pJo  equivalency 


continuing  suitability  and 
effectiveness.  Records  of  such 
reviews  shall  be  maintained  (see 
4.16) . 

Note  -  Management  reviews 
normally  include  assessment  of 
the  results  of  internal  quality 
audits,  but  are  carried  out  by, 
or  on  behalf  of,  the  supplier's 
management,  viz  management 
personnel  having  direct 
responsibility  for  the  system. 
(See  4.17) 


4.2  Quality  system 

The  supplier  shall  establish  and 
maintain  a  documented  quality 
system  as  a  means  of  ensuring 
that  product  conforms  to 
specified  requirements.  This 
shall  include 

a)  the  preparation  of 
documented  quality  system 
procedures  and  instructions  in 
accordance  with  the  requirements 
of  this  international  standard; 

b)  the  effective 
implementation  of  the  documented 
quality  system  procedures  and 
instructions . 

Note  -  In  meeting  specified 
requirements,  timely 
consideration  needs  to  be  given 
to  the  following  activities: 

a)  the  preparation  of  quality 
plans  and  a  quality  manual  in 
accordance  with  the  specified 
requirements ; 

b)  the  identification  and 
acquisition  of  any  controls, 
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rocesses,  inspection  equipment, 
fixtures,  total  production 
resources  and  skills  that  may  be 
needed  to  achieve  the  required 
quality . ; 

c)  the  updating,  as  necessary, 
of  quality  control,  inspection 
and  testing  techniques, 
including  the  development  of  new 
instrumentation; 

d)  the  identification  of  any 
measurement  requirement 
involving  capability  that 
exceeds  the  kown  state  of  the 
art  in  sufficient  time  for  the 
needed  capability  to  be 
developed; 

e)  the  clarification  of 
standards  of  accepatability  for 
all  features  and  requirements, 
including  those  which  contain  a 
subjective  element; 

f)  the  compatibility  of  the 
design,  the  production  process, 
installation,  inspection  and 
test  procedures  and  the 
applicable  documentation; 

g)  the  identification  and 
preparation  of  qualtity  records 
(see  4.16) . 


Contract  review 


The  supplier  shall  establish  and 
maintain  procedures  for  contract 
review  and  for  the  coordination 
of  these  activities. 

Each  constract  shall  be  reviewed 
by  the  supplier  to  ensure  that 

a)  the  requirements  are 
adequately  defined  and 
documented; 


responsible  for  compliance  with 
all  provisions  of  the  contract 
and  for  furnishing  specified 
supplies  and  services  which  meet 
all  the  requirements  fo  the 
contract.  If  any  inconsistency 
exists  between  the  contract 
schedule  or  its  general 
provisions  and  this 
specification,  the  contract 
schedule  and  the  general 
provisions  shall  control.  The 
contractor's  quality  program 
shall  be  planned  and  used  in  a 
manner  to  support  reliability 
effectively . 


4.1  Drawings,  Documentation  an 
Changes.  A  procedure  shall  be 
maintained  that  concerns  itself 
with  the  adequacy,  the 
completeness  and  the  currentness 
of  drawings  and  with  the  control 
of  changes  in  design.  With 
respect  to  the  currentness  of 
drawings  and  changes,  the 
contractor  shall  assure  that 
requirements  for  the  effectivity 
point  of  drawings  and  changes 
are  met  and  that  obsolete 
drawings  and  change  requirements 
are  removed  from  all  points  of 
issue  and  use.  Some  means  of 
recording  the  effective  points 
shall  be  employed  and  be 
available  to  the  Government. 

With  respect  to  design 
drawings  and  design 
specifications,  a  procedure 
shall  be  maintained  that  shall 
provide  for  the  evaluation  of 
the  engineering  adequacy  and  an 
evaluation  of  the  adequacy  of 
proposed  changes .  The 
evaluation  shall  encompass  both 
the  adequacy  in  relation  to 
standard  engineering  and  design 
practices  and  the  adequacy  with 
respect  to  the  design  and 
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4.4  Design  Contro 

4.4.1  General 

The  supplier  shall  establish  and 
maintain  procedures  to  control 
and  verify  the  design  of  the 
product  in  order  to  ensure  that 
the  specified  requirements  are 
met . 

4.4.2  Design  and  development 
planning 

The  supplier  shall  draw  up  plans 
that  identify  the  responsibility 
for  each  design  and  development 
activity.  The  plans  shall 
describe  or  reference  these 
activities  and  shall  be  updated 
as  the  design  evolves. 

4. 4. 2.1  Activity  assignment 

The  design  and  verification 
activities  shall  be  planned  and 
assigned  to  qualified  personnel 
equipped  with  adequate 
resources . 

4. 4. 2. 2  Organizational  and 
technical  interfaces 
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purpose  of  the  product  to  which 
the  drawing  relates. 

With  respect  to  supplemental 
specif ications ,  process  ins¬ 
tructions,  production  engineer¬ 
ing  instructions,  industrial 
engineering  instructions  and 
work  instructions  relating  to  a 
particlar  design,  the  contractor 
shall  be  responsible  for  a 
review  of  thier  adequacy, 
currentness  and  completeness. 

The  quality  program  must  provide 
complete  coverage  of  all 
information  necessary  to  produce 
an  article  in  complete 
conformity  with  requirements  of 
the  design. 

The  quality  program  shall 
assure  that  there  is  complete 
compliance  with  contract 
requirements  for  proposing, 
approving,  and  effecting  of  eng¬ 
ineering  changes.  The  quality 
program  shall  provide  for 
monitoring  effectively  the 
drawing  changes  of  lesser 
importance  not  requiring 
approval  by  Government  design 
authorities . 

Delivery  of  correct  drawings 
and  change  information  to  the 
Government  in  connection  with 
data  acquisition  shall  be  an 
integral  part  of  the  quality 
program.  This  includes  full 
compliance  with  contract 
requirements  concerning  rights 
and  data  both  proprietary  and 
other.  The  quality  program's 
responsibility  for  drawings  and 
changes  extend  to  the  drawings 
and  changes  provided  by  the  sub- 
contracors  and  vendors  for  the 
contract . 


between  different  groups  shall 
be  identified  and  the  necessary 
information  documented, 
transmitted  and  regularly 
reviewed. 

4.4.3  Design  input 

Design  input  requirements  relat¬ 
ing  to  the  product  shall  be 
identified,  documented  and  their 
selection  reviewed  by  the 
supplier  for  adequacy. 

Incomplete,  ambiguous  or  con¬ 
flicting  requirements  shall  be 
resolved  with  those  responsible 
for  drawing  up  these 
requirements . 

4.4.4  Design  output 

Design  output  shall  be 
documented  and  expressed  in 
terms  of  requirements, 
calculations  and  analyses. 

Design  output  shall 

a)  meet  the  design  input 
requirements ; 

b)  contain  or  reference 
acceptance  criteria; 

c)  conform  to  appropriate 
regulatory  requirements  whether 
or  not  these  have  been  stated  in 
the  input  information; 

d)  identify  those 
characteristics  of  the  design 
that  are  crucial  to  the  safe  and 
proper  functioning  of  the 
product . 

4.4.5  Design  verification 

The  supplier  shall  plan, 
estabish,  document  and  assign  to 
competent  personnel  functions 
for  verifying  the  design. 
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Design  verification  shall 
establish  that  design  output 
meets  the  design  input  require¬ 
ment  (see  4.4.4)  by  means  of 
design  control  measures  such  as: 

a)  holding  and  recording 
design  reviews  (see  4.16); 

b)  undertaking  qualification 
tests  and  demonstrations; 

c)  carry  out  alternative 
calculations ; 

d)  comparing  new  design  with  a 
similar  proven  design,  if 
available . 

4.4.6  Design  changes 

The  supplier  shall  establish  and 
maintain  procedures  for  the 
identification,  documentation 
and  appropriate  review  and 
approval  of  all  changes  and 
modifications. 


Document  contro 

4.5.1  Document  approval  and 
issue 


The  supplier  shall  establish  and 
maintain  procedures  to  control 
all  documents  and  data  that 
relate  to  the  requirements  of 
this  International  Standard. 
These  documents  shall  be  re¬ 
viewed  and  approved  for  adequacy 
by  authorized  personnel  prior  to 
issue.  This  control  shall  ensure 
that 

a)  the  pertinent  issues  of 
appropriate  documents  are 
available  at  all  locations  where 
operations  essential  to  the 
effective  functioning  of  the 
quality  system  are  performed; 


b)  obsolete  documents  are 
promptly  removed  from  all  points 
of  issue  or  use. 


.  CONTROL  OF  PURCHASES 

5.1  Responsibility.  The 

contractor  is  responsible  for 
assuring  that  all  supplies  and 
services  procured  from  his 
suppliers  (subcontractos  and 
vendors)  conform  to  the  contract 
requirements.  The  selection  of 
sources  and  the  nature  and 
extent  of  control  exercised  by 
the  contractor  shall  be 
dependent  upon  the  type  of  sup¬ 
plies,  his  supplier's  demon¬ 
strated  capability  to  perform, 


4.5.2  Document  changes/ 
modifications 

Changes  to  documents  shall  be 
reviewed  and  approved  by  the 
same  functions/organizations 
that  performed  the  original 
review  and  approval  unless 
specifically  designated  other¬ 
wise.  The  designated  organiz¬ 
ations  shall  have  access  to 
pertinent  background  information 
upon  which  to  base  their  review 
and  approval . 

Where  practicable,  the  nature  of 
the  change  shall  be  identified 
in  the  document  or  the 
appropriate  attachments. 

A  master  list  or  equivalent 
document  control  procedure  shall 
be  established  to  identify  the 
current  revision  of  documents 
inthe  use  of  non-applicable 
documents . 

Documents  shall  be  reissued 
after  a  practical  number  of 
changes  have  been  made . 


.6  Purchasing 

4.6.1  General 

The  supplier  shall  ensure  that 
purchased  product  conforms  to 
specified  requirements. 

4.6.2  Assessment  of  sub¬ 
contractors 

The  supplier  shall  select  sub¬ 
contractors  on  the  basis  of 
their  ability  to  meet  sub¬ 
contract  requirements,  includ- 


~  and  the  quality  evidence  made 
available.  To  assure  an  adequate 
and  economical  control  of  such 
material,  the  contractor  shall 
utilize  to  the  fullest  extent 
objective  evidence  of  quality 
furnished  by  his  suppliers.  When 
the  Government  elects  to  perform 
inspection  at  a  supplier's 
plant,  such  inspection  shall  not 
be  used  by  contractors  as  evid¬ 
ence  of  effective  control  of 
quality  by  such  suppliers.  The 
inclusion  of  a  product  on  the 
Qualified  Products  List  only 
signifies  that  at  one  time  the 
manufacturer  made  a  product 
which  met  specification  require¬ 
ments.  It  does  not  relieve  the 
contractor  of  his  responsibility 
for  furnishing  supplies  that 
meet  all  specif ication  require¬ 
ments  or  for  the  performance  of 
specified  inspections  and  tests 
for  such  material.  The  effect¬ 
iveness  and  integrity  of  the 
control  of  quality  by  his 
suppliers  shall  be  assessed  and 
reviewed  by  the  contractor  at 
intervals  consistent  with  the 
complexity  and  quantity  of 
product.  Inspection  of  products 
upon  delivery  to  the  contractor 
shall  be  used  for  assessment  and 
review  to  the  extent  necessary 
for  adequate  assurance  of 
quality.  Test  reports,  inspect¬ 
ion  records,  certificates  and 
other  suitable  evidence  relating 
to  the  supplier's  control  of 
quality  should  be  used  in  the 
contractor's  assessment  and 
review.  The  contractor's 
responsibility  for  the  control 
of  purchases  includes  the 
establishment  of  a  procedure  for 
(1)  the  selection  of  qualified 
suppliers,  (2)  the  transmission 
of  applicable  design  and  quality 
requirements  in  the  Government 
contracts  and  associated  tech- 


ing  quality  requirements.  The 
supplier  shall  establish  and 
maintain  records  of  acceptable 
sub-contractors  (see  4.16). 

The  selection  of  sub¬ 
contractors,  and  the  type  and 
extent  of  control  exercised  by 
the  supplier,  shall  be  dependent 
upon  the  type  of  product  and, 
where  appropriate,  on  records  of 
subcontractor's  previously 
demonstrated  capability  and 
performance . 

The  supplier  shall  ensure  that 
quality  system  controls  are 
effective . 
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nical  requirements,  (3)  the 
evaluation  of  the  adequacy  of 
procured  items,  and  (4) 
effective  provisions  for  early 
information  feedback  and 
correction  of  nonconformances. 


Purchasing  data 


Purchasing  documents  shall 
tor  contain  data  clearly  describing 
s  a  the  product  ordered,  including, 
rol  Iwhere  applicable, 

a)  the  type,  class,  style, 
grade  or  other  precise 
identification; 

b)  the  title  or  other 
positive  identification,  and 
applicable  issue  of  specific¬ 
ations,  drawings,  process  re¬ 
quirements,  inspection 
instructions  and  other  relevant 
technical  data,  including 
requirements  for  approval  or 
qualification  of  product, 
procedures,  process  equipment 
and  personnel; 

c)  the  title,  number  and  issue 
of  the  quality  system 
International  Standard  to  be 
applied  to  the  product. 

The  supplier  shall  review  and 
approve  purchasing  documents  for 
adequacy  of  specified 
requirements  prior  to  release. 

4.6.4  Verification  of  purchased 
product 

Where  specified  in  the  contract, 
the  purchaser  or  his 
representative  shall  be  afforded 
the  right  to  verify  at  source  or 
upon  receipt  that  purchased 
iroduct  conforms  to  specified 


or  vendor  source  when  such 
action  is  necessary  to  assure 
that  the  contractor's  respon¬ 
sibility  for  complete  assurance 
of  product  quality.  Require¬ 
ments  shall  be  included  for 
chemical  and  physical  testing 
and  recording  in  connection  with 
the  purchase  of  raw  materials  by 
his  suppliers.  The  purchase 
orders  must  also  contain  a 
requirement  for  such  suppliers 
to  notify  and  obtain  approval 
from  the  contractor  of  changes 
in  design  of  the  products. 
Necessary  instructions  should  be 
provided  when  provision  is  made 
for  direct  shipment  from  the 
sub-contractor  to  Government 
activities . 

7.2  Government  Property. 

7.2.1  Government -furnished 
Material. 

When  material  is  furnished  by 
the  Government,  the  contractor's 
procedures  shall  include  at 
least  the  following: 

a)  Examination  upon  receipt, 
consistent  with  practicability 
to  detect  damage  in  transit; 

b)  Inspection  for  completeness 
and  proper  type; 

c)  Periodic  inspection  and 
precautions  to  assure  adequate 
storage  conditions  and  to  guard 
against  damage  from  handling  and 
deterioration  during  storage; 

d)  Functional  testing,  either 
prior  to  or  after  installation, 
or  both,  as  required  by  contract 
to  determine  satisfactory 
operation; 

e)  Identification  and  protect¬ 
ion  from  improper  use  or  dis¬ 
position;  and 

f)  Verification  of  quantity. 


requirements.  Verification  by 
the  purchaser  shall  not  absolve 
the  supplier  of  the 
responsibility  to  provide 
acceptable  product  nor  shall  it 
preclude  subsequent  rejection. 
When  the  purchaser  or  his 
representative  elects  to  carry 
out  verification  at  the  sub¬ 
contractor's  plant,  verification 
shall  not  be  used  by  the 
supplier  as  evidence  of  effecive 
control  of  quality  by  the  sub¬ 
contractor  . 


i.l  Purchaser  supplied  product 

The  supplier  shall  establish  and: 
maintain  procedures  for 
verification,  storage  and  main¬ 
tenance  of  purchaser  supplied 
product  provided  for  incorp¬ 
oration  into  the  supplies.  Any 
such  product  that  is  lost, 
damaged  or  is  otherwise  unsuit¬ 
able  for  use  shall  be  recorded 
and  reported  to  the  purchaser 
( see  4.16) . 

Note  -  Verification  by  the 
supplier  does  not  absolve  the 
purchaser  of  the  responsibility 
to  provide  acceptable  product. 

found  damaged,  malfunctioning, 
or  otherwise  unsuitable  for  use. 
In  the  event  of  damage  or  mal- 
fucntioning  during  or  after 
installation,  the  contractor 
shall  determine  and  record 
probable  cause  and  necessity  for 
withholding  material  from  use. 
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7.2.2  Damaged  Government - 
furnished  Material .  The 

contractor  shall  report  to  the 
Government  Representative 
anyGovernment- furnished  material 

No  equivalency 

4.8  Product  identification  and 
traceability 

Where  appropriate,  the  supplier 
shall  establish  and  maintain 
procedures  for  identifying  the 
product  from  applicable 
drawings,  specif icaitons  or 
other  documents,  during  all 
stages  of  production,  delivery 
and  installation. 

Where,  and  to  the  extent  that, 
traceability  is  a  specified 
requirement,  individual  product 
or  batches  shall  have  a  unique 
identification.  This  identif¬ 
ication  shall  be  recorded  (see 
4.16)  . 

6.2  Production  Processing  and 
Fabrication.  The  contractor's 
quality  program  must  assure  that 
all  machining,  wiring,  batching, 
shaping  and  all  basic  production 
operations  of  any  type  is 
accomplished  under  controlled 
conditions.  Controlled  condit¬ 
ions  include  documented  work 
instructions,  adequate  product¬ 
ion  equipment,  and  any  special 
working  environment.  Documented 
work  instructions  are  considered 
to  be  the  criteria  for  much  of 
the  production,  processing  and 
fabrication  work.  These 
instructions  are  the  criteria 
for  acceptable  or  unacceptable 
"workmanship" .  The  quality 
program  will  effectively  monitor 
the  issuance  of  and  compliance 
with  all  of  these  work 
instructions . 

Physical  examination,  measure¬ 
ment  or  tests  of  the  material  or 

4 . 9  Process  control 

4.9.1  General 

The  supplier  shall  identify  and 
plan  the  production  and,  where 
applicable,  installation 
processes  which  directly  affect 
quality  and  shall  ensure  that 
these  processes  are  carried  out 
under  controlled  conditions. 
Controlled  conditions  shall 
include  the  following: 

a)  documented  work 
instructions  defining  the  manner 
of  production  and  installation, 
where  the  absence  of  such 
instructions  would  adversely 
affect  quality,  use  of  suitable 
production  and  installation 
equipment,  suitable  working 
environment,  compliance  with 
reference  standards/codes  and 
quality  plans; 
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products  processed  is  necessary 
for  each  work  operation  and  must 
also  be  conducted  under 
controlled  conditions.  If 
physical  inspection  of  processed 
manterial  is  impossible  or 
disadvantageous,  indirect 
control  by  monitoring  processing 
methods,  equipment  and  personnel 
shall  be  provided.  Both  physical 
inspection  and  process 
monitoring  shall  be  provided 
when  control  is  inadequate 
without  both,  or  when  contract 
or  specification  requires  both. 

Inspection  and  monitoring  of 
processed  material  or  produces 
shall  be  accomplished  in  any 
suitable  manner  selected  by  the 
contractor.  Methods  of 
inspection  and  monitoring  shall 
be  corrected  any  time  their 
unsuitability  with  reasonable 
evidence  is  demonstrated. 
Adherence  to  selected  methods 
for  inspection  and  monitoring 
shall  be  complete  and  continous. 
Corrective  measures  shall  be 
takentaken  when  noncompliance 
occurs . 

Inspection  by  machine 
operators,  automated  inspection 
gages,  moving  line  or  lot 
sampling,  setup  or  first  piece 
approval,  production  line 
inspection  station,  inspection 
or  test  department,  roving 
inspectors  -  any  other  type  of 
inspection  -  shall  be  employed 
in  any  combination  desired  by 
the  contractor  which  will 
adequately  and  efficiently 
protect  product  quality  and  the 
integrity  of  processing. 

Criteri ^  for  approval  and 
rejection  shall  be  provided  for 
all  inspection  of  product  and 
monitoring  of  methods,  equip¬ 
ment,  and  personnel.  Means  for 


b)  monitoring  and  control  of 
suitable  process  and  product 
characteristics  during 
production  and  installation; 

c)  the  approval  of  processes 
and  equipment,  as  appropriate; 

d)  criteria  for  workmanship 
which  shall  be  stipulated,  to 
the  greatest  practicable  extent, 
in  written  standards  by  means  of 
representative  samples. 


4.9.2  Special  processes 

These  are  processes,  the  results 
of  which  cannot  be  fully 
verified  by  subsequent 
inspection  and  testing  of  the 
product  and  where,  for  example, 
processing  deficiencies  may 
become  apparent  only  after  the 
product  is  in  use.  Accordingly, 
continuous  monitoring  and/or 
compliance  with  documented 
procedures  is  required  to  ensure 
that  che  specified  requirements 
are  met.  These  processes  shall 
be  qualified  and  shall  also 
comply  with  requirements  of 
4.9.1. 

Records  shall  be  maintained  for 
qualified  processes,  equipment 
and  personnel,  as  appropriate. 
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Inspection  and  testing 

4.10.1  Receiving  inspection  and 
testing 

4.10.1.1  The  supplier  shall 
ensure  that  incoming  product  is 
not  used  or  processed  (except  in 
the  circumstances  described  in 
4.10.2)  until  it  has  been 
inspected  or  otherwise  verified 
as  conforming  to  specified 
requirements.  Verification  shall 
be  in  accordance  with  the 
quality  plan  or  documented 
procedures . 

4.10.1.2  Where  incoming  product 
is  released  for  urgent  product¬ 
ion  purposes,  it  shall  be  pos¬ 
itively  identified  and  recorded 
(see  4.16)  in  order  to  permit 
immediate  recall  and  replace¬ 
ment  in  the  event  of  noncon¬ 
formance  to  specified  require¬ 
ments  . 

Note  -  In  determining  the  amount 
and  nature  of  receiving 
inspection,  consideration  should 
be  given  to  the  control 
exercised  at  source  and  doc¬ 
umented  evidence  of  quality 
conformance  provided. 
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Excerpts  from  6.2  Production 
Processing  and  Fabrication 

Physical  examination,  measure¬ 
ment  or  tests  of  the  material  or 
products  processed  is  necessary 
'for  each  work  operation  and  must 
also  be  conducted  under 
controlled  conditions.  If 
hysical  inspection  of  processed 
aterial  is  impossible  or  disad- 
antageous,  indirect  control  by 
onitoring  porcessing  methods, 
equipment  and  personnel  shall  be 
provided.  Both  physical  inspect¬ 
ion  and  process  monitoring  shall 
be  provided  when  control  is 
inadequate  without  both,  or  when 
contract  or  specification 
requires  both. 

Inspection  by  machine 
operators,  automated  inspection 
gages,  moving  line  or  lot 
sampling,  setup  or  first  piece 
approval,  production  line 
inspection  station,  inspection 
or  test  department,  roving 
inspectors  -  any  other  type  of 
inspection  -  shall  be  employed 
in  any  combination  desired  by 
the  contractor  which  will 
adequately  and  efficiently 
protect  product  quality  and  the 
integrity  of  processing. 

Criteria  for  approval  and 
rejection  shall  be  provided  for 
all  inspection  of  product  and 
monitoring  of  methods,  equip¬ 
ment,  and  personnel.  Means  for 
identifying  approved  and 
rejected  product  shall  be 
rovided . 


.3  Completed  Item  Inspection 
and  Testing.  The  quality  program 
shall  assure  that  there  is  a 
system  for  final  inspection  and 
test  of  completed  products.  Such 
testing  shall  provide  a  measure 
of  the  overall  quality  of  the 
completed  product  and  shall  be 


4.10.2  In-process  inspection  an 
testing 

The  supplier  shall 

a)  inspect,  test  and  identify 
product  as  required  by  the 
quality  plan  or  documented 
procedures ; 

li 

b)  establish  product  conform¬ 
ance  to  specified  requirements 
by  use  of  process  monitoring  and 
control  methods; 

c)  hold  product  until  the 
required  inspection  and  tests 
have  been  completed  or  nec¬ 
essary  reports  have  been  re¬ 
ceived  and  verified  except  when 
product  is  released  under  pos¬ 
itive  recall  procedures  (see 
4.10.1).  Release  under  positive 
recall  procedures  shall  not 
preclude  the  activities  outlined 
in  4.10.2a); 

d)  identify  nonconforming 
product . 
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performed  so  that  it  simulates, 
to  a  sufficient  degree,  product 
end  use  and  functioning.  Such 
simulation  frequently  involves 
appropriate  life  and  endurance 
tests  and  qualification  testing. 
Final  inspection  and  testing 
shall  provide  for  reporting  to 
designers  any  unusual  difficul¬ 
ties,  deficiencies  or  question¬ 
able  conditions.  When  modific¬ 
ations,  repairs  or  replacements 
are  required  after  final  in¬ 
spection  or  testing,  there  shall 
be  reinspection  and  retesting  of 
any  characteristics  affected. 


ied  either  on  receipt  of  pro¬ 
duct  or  in-process,  have  been 
carried  out  and  that  the  data 
meets  specified  requirements. 

The  supplier  shall  carry  out  all 
final  inspection  and  testing  in 
accordance  with  the  quality  plan 
or  documented  procedures  to 
complete  the  evidence  of 
conformance  of  the  finished 
product  to  the  spec-ified 
requirements . 

No  product  shall  be  despatched 
until  all  the  activities  spec¬ 
ified  in  the  quality  plan  or 
documented  procedures  have  been 
satisfactorily  completed  and  the 
associated  data  and  doc¬ 
umentation  is  available  and 
authorized . 


4.10.4  Inspection  and  test 
records 


the  supplier  shall  establish  and 
maintain  records  which  give 
evidence  that  the  product  has 
passed  inspection  and/or  test 
with  defined  acceptance  criteria 
( see  4.16) . 


4.11  Inspection,  measuring  and 
test  equipment 

The  supplier  shall  control, 
calibrate  and  maintain  inspect¬ 
ion,  measuring  and  test  equip¬ 
ment,  whether  owned  by  the  sup¬ 
plier,  on  loan,  or  provided  by 
the  purchaser,  to  demonstrate 
the  conformance  of  product  to 
the  specified  requirements. 
Equipment  shall  be  used  in  a 
manner  which  ensures  that  meas¬ 
urement  uncertainty  is  known  and 
is  consistent  with  the  required 
measurement  cap-ability. 
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conformity  with  military  spec- 
ification  MIL-C-45662.  In  add¬ 
ition,  the  contractor  shall 
insure  the  use  of  only  such  sub¬ 
contractor  and  vendor  sources 
that  depend  upon  calibration 
systems  which  effectively 
control  the  accuracy  of 
measuring  and  testing  equipment. 


The  supplier  shall 

a)  identify  the  measurements 
to  be  made,  the  accuracy 
required  and  select  the  approp¬ 
riate  inspection,  measuring  and 
test  equipment; 

b)  identify,  calibrate  and 
adjust  all  inspection,  meas¬ 
uring  and  test  equipment  and 
devices  that  can  affect  product 
quality  at  prescribed  inter¬ 
vals,  or  prior  to  use,  against 
certified  equipment  having  a 
known  valid  relationship  to 
nationally  recognized  standards 
-  where  no  such  standards  exist, 
the  basis  used  for  cal-ibration 
shall  be  documented; 

c)  establish,  document  and 
maintain  calibration  proced¬ 
ures,  including  details  of 
equipment  type,  identification 
number,  location,  frequency  of 
checks,  check  method,  accept¬ 
ance  criteria  and  the  action  to 
be  taken  when  results  are  un¬ 
satisfactory; 

d)  ensure  that  the  inspect¬ 
ion,  measuring  and  test  equip¬ 
ment  is  capable  of  the  accuracy 
and  precision  necessary; 

e)  identify  inspection, 
meas-uring  and  test  equipment 
with  a  suitable  indicator  or 
approved  identification  record 
to  show  the  calibration  status; 

f)  maintain  calibration 
records  for  inspection,  meas¬ 
uring  and  test  equipment  (see 
4.16); 

g)  assess  and  document  the 
validity  of  previous  inspect-ion 
and  test  results  when  in¬ 
spection,  measuring  and  test 
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h)  ensure  that  the  environ- 
ental  conditions  are  suitable 

for  the  calibration,  inspect¬ 
ions,  measurements  and  tests 
being  carried  out; 

i)  insure  that  the  handling, 
preservation  and  storage  of 
inspection,  measuring  and  test 
equipment  is  such  that  the 
accuracy  and  fitness  for  use  is 
maintained; 

j)  safeguard  inspection, 
measuring  and  test  facilities, 
including  both  test  hardware  and 
test  software,  from  adjust-ments 
which  would  invalidate  the 
calibration  setting. 
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Inspection  and  test  status 


The  inspection  and  test  status 
of  product  shall  be  identified 
by  using  markings,  authorized 
stamps,  tags,  labels,  routing 
cards,  inspection  records,  test 
software,  physical  location  or 
other  suitable  means,  which 
indicate  the  conformance  or 
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Corrective  Action.  The 

quality  program  shall  detect 
promplty  and  correct  assignable 
conditions  adverse  to  quality. 
Design,  purchasing,  manufactur¬ 
ing,  testing  or  other  operations 
which  could  result  in  or  have 
resulted  in  defective  supplies, 
services,  facilities,  technical 
data,  standards  or  other 
elements  of  contract  performance 
which  could  create  excessive 
losses  or  costs  must  be  identi¬ 
fied  and  changed  as  a  result  of 
the  quality  program.  Corrective 
action  will  extend  to  the  per¬ 
formance  of  all  suppliers  and 
vendors  and  will  be  responsive 
to  data  and  product  forwarded 
from  users.  Corrective  action 
shall  include  as  a  minimum: 

a)  Analysis  of  data  and  exam¬ 
ination  of  product  scrapped  or 
reworked  to  determine  extent  and 
causes ; 

b)  Analysis  of  trends  in  pro¬ 
cesses  or  performance  of  work  to 

revent  nonconforming  product; 


c)  re-graded  for  alternative 
applications,  or 

d)  rejected  or  scrapped. 

Where  required  by  the  contract, 
the  proposed  use  or  repair  of 
product  (see  4.13.1b)  which  does 
not  conform  to  specified 
requirements  shall  be  reported 
for  concession  to  the  purchaser 
or  his  representative.  The 
description  of  nonconformity 
that  has  been  accepted,  and  of 
repairs,  shall  be  recorded  to 
denote  the  actual  condition  (see 
4.16)  . 

Repaired  and  reworked  product 
shall  be  re-inspected  in 
accordance  with  documented 
rocedures . 


.14  Corrective  action 

The  supplier  shall  establish, 
document  and  maintain  procedures 
for 

a)  investigating  the  cause  of 
nonconforming  product  and  the 
corrective  action  needed  to 
prevent  recurrence; 

b)  analysing  all  processes, 
work  operations,  concessions, 
quality  records,  service  re¬ 
ports  and  customer  complaints  to 
detect  and  eliminate  potent-ial 
causes  of  nonconforming  product; 

c)  initiating  preventive 
actions  to  deal  with  problems  to 
a  level  corresponding  to  the 
risks  encountered; 

d)  applying  controls  to  ensure 
that  corrective  actions  are 
taken  and  that  they  are  ef¬ 
fective; 
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Where  appropriate,  the  supplier 
shall  establish  procedures  for 
identifying  adequate  statist¬ 
ical  techniques  required  for 
verifying  the  acceptability  of 
process  capability  and  product 
characteristic 
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subject  to  review  by  the 
cognizant  Government  Represent¬ 
ative.  Any  sampling  plan  used 
shall  provide  valid  confidence 
and  quality  levels. 

No  equivaiancy 
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Internal  quality  audits 


The  supplier  shall  carry  out  a 
comprehensive  system  of  planned 
and  documented  internal  quality 
audits  to  verify  whether  quality 
activities  comply  with  planned 
arrangements  and  to  determine 
the  effectiveness  of  the  quality 
system. 

Audits  shall  be  scheduled  on  the 
basis  of  the  status  and 
importance  of  the  activity. 

The  audits  and  follow-up  actions 
shall  be  carried  out  in 
accordance  with  documented 
procedures . 

The  results  of  the  audits  shall 
be  documented  and  brought  to  the 
attention  of  the  personnel 
having  responsibility  in  the 
area  audited.  The  management 
personnel  responsible  for  the 
area  sahll  take  timely 
corrective  action  on  the 
deficiencies  found  by  the  audit 
( see  4.1.3)  . 
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3.6  costs  Related  to  Quality!" 

The  contractor  shall  maintain 
jand  use  quality  cost  data  as  a 
management  element  of  the 
jquality  program.  These  data 
shall  serve  the  purpose  of  id¬ 
entifying  the  cost  of  both  the 
prevention  and  correction  of 
nonconforming  supplies  (e.g., 
laobr  and  material  involved  in 
material  spoilage  caused  by 
defective  work,  correction  of 
defective  work  and  for  quality 
control  exercised  by  the  con¬ 
tractor  at  subcontractor's  or 
(vendor's  facilities).  The 
specified  quality  cost  data  to 
be  maintained  and  used  will  be 
determined  by  the  contractor. 
These  data  shall,  on  request,  be 
identified  and  made  available 
for  "on  site"  review  by  the 
|Government  Representative. 

.4  Use  of  Contractor's  Inspect - 
ion  Equipment.  The  contracrtor ' s 
gages,  measuring  and  testing 
[devices  shall  be  made  available 
for  use  by  the  Government  when 
required  to  determine  conform- 
|ance  with  contract  requirements. 
If  conditions  warrant,  contract¬ 
or's  personnel  shall  be  made 
available  for  operation  of  such 
devices  and  for  verification  of 
their  accuracy  and  condition. 


No  equivalency 


No  equivalency 
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7 . 1  Government  Inspection  at 
Subcontractor  of  Vendor 
Facilities.  The  Governments 
reserves  the  right  to  inspect  at 
source  supplies  or  services  not 
manufactured  or  performed  with 
the  contractor's  facility. 
Government  inspection  shall  not 
constitute  accepance;  nor  shall 
it  in  any  way  replace  contractor 
inspection  or  otherwise  relieve 
the  contractor  of  his  respons¬ 
ibility  to  furnish  an  acceptable 
end  item.  The  purpose  of  this 
inspection  is  to  assist  the 
Government  Representative  at  the 
contractor's  facility  to  deter¬ 
mine  the  conformance  of  supplies 
or  services  with  contract  re¬ 
quirements.  Such  inspection  can 
only  be  requested  by  or  under 
authorization  of  the  Government 
Representative.  When  Government 
inspection  is  required,  the 
contractor  shall  add  to  his 
purchasing  document  the 
following  statement: 

"Government  inspection  is 
required  prior  to  ship¬ 
ment  from  your  plant. 

Upon  receipt  of  this  or¬ 
der,  promptly  notify  the 
Government  Representative 
who  normally  services 
your  plant  so  that  ap¬ 
propriate  planning  for 
Government  inspection 
can  be  accomplished." 

When,  under  authorization  from 
the  Government  Representative, 
copies  of  the  purchasing 
document  are  to  be  furnished 
directly  by  the  subcontractor  or 
vendor  to  the  Government  Rep¬ 
resentative  at  his  facility 
rather  than  through  Government 
Channels,  the  contractor  shall 
add  to  his  purchasing  document  a 
statement  substantially  as 
follows : 
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"On  receipt  of  this  order, 
promptly  furnish  a  copy  to 
the  Government  representative 
who  normally  services  your 
plant,  or,  if  none,  to  the 
nearest  Army,  Navy,  Air 
Force,  or  Defense  Supply  Ag¬ 
ency  inspection  office.  In 
the  event  the  representative 
or  office  cannot  be  located, 
our  purchasing  agent  should 
be  notified  immediately." 

All  documents  and  referenced 
data  for  purchases  applying  to  a 
Government  contract  shall  be 
available  for  review  by  the  Gov¬ 
ernment  Representative  to  deter¬ 
mine  compliance  with  the  re¬ 
quirements  for  the  control  of 
such  purchases.  Copies  of  pur¬ 
chasing  documents  for  Government 
purposes  shall  be  furnished  in 
accordance  with  the  instructions 
of  the  Government  Representat¬ 
ive.  The  contractor  shall  make 
available  to  the  Government 
Representative  reports  of  any 
nonconformance  found  on  Govern¬ 
ment  source  inspected  supplies 
and  shall  (when  requested) 
require  the  supplier  to 
coordinate  with  his  Government 
Representative  on  corrective 
action . 


115 


APPENDIX  C 


COMPARISON  OF  MIL-I-45208  TO  ISO  9002 


MIL-I-45208 

1.1  Scope.  This  specification 
establihes  requirements  for 
contractors'  inspection  systems. 
These  requirements  pertain  to 
the  inspections  and  tests  nec¬ 
essary  to  substantiate  product 
conformance  to  drawings , spec¬ 
ifications  and  contract  require¬ 
ments  and  to  all  inspections  and 
tests  required  by  the  contract. 
These  requirements  are  in  add¬ 
ition  to  those  inspections  and 
tests  set  forth  in  applicable 
specifications  and  other  con¬ 
tractual  documents. 


ISO  9002 

4  Quality  system  requirements 

4.1  Mangement  responsibility 

4.1.1.  Quality  policy 

The  supplier's  management  shall 
define  and  document  its  policy 
and  objectives  for,  and 
commitment  to,  quality.  The 
supplier  shall  ensure  that  this 
policy  is  understood, 
implemented  and  maintained  at 
all  levels  in  the  organization. 
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contract.  The  Government  at  its 
option  may  furnish  written  of 
the  acceptability  or  nonaccept¬ 
ability  of  the  inspection 
system.  The  contractor  shall 
notify  the  Government  Represent¬ 
ative  in  writing  of  any  change 
to  his  inspection  system.  The 
inspection  system  shall  be  sub¬ 
ject  to  disapproval  if  changes 
thereto  would  result  in  non- 
conforming  product . 

No  equivalency 


e)  control  further  processing, 
delivery  or  installation  of 
nonconforming  product  until  the 
deficiency  or  unsatisfactory 
condition  has  been  corrected. 


Management  review 


The  quality  system  adopted  to 
satisfy  the  requirements  of  this 
International  Standard  shall  be 
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No  equivalency 


reviewed  at  appropriate 
intervals  by  the  supplier's 
management  to  ensure  its 
continuing  suitability  and 
effectiveness.  Records  of  such 
reviews  shall  be  maintained  (see 
4.16). 

Note  -  Management  reviews 
normally  include  assessment  of 
the  results  of  internal  quality 
audits,  but  are  carried  out  by, 
or  on  behalf  of,  the  supplier's 
management,  viz  management 
personnel  having  direct 
responsibility  for  the  system. 
(See  4.17) 

4.2  Quality  system 

The  supplier  shall  establish  and 
maintain  a  documented  quality 
system  as  a  means  of  ensuring 
that  product  conforms  to 
specified  requirements.  This 
shall  include 

a)  the  preparation  of 
documented  quality  system 
procedures  and  instructions  in 
accordance  with  the  requirements 
of  this  international  standard; 

b)  the  effective 
implementation  of  the  documented 
quality  system  procedures  and 
instructions . 

Note  -  In  meeting  specified 
requirements,  timely 
consideration  needs  to  be  given 
to  the  following  activities: 

a)  the  preparation  of  quality 
plans  and  a  quality  manual  in 
accordance  with  the  specified 
requirements ; 

b)  the  identification  and 
acquisition  of  any  controls, 
processes,  inspection  equipment, 
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fixtures,  total  production 
resources  and  skills  that  may  be 
needed  to  achieve  the  required 
quality . ; 

c)  the  updating,  as  necessary, 
of  quality  control,  inspection 
and  testing  techniques, 
including  the  development  of  new 
instrumentation; 

d)  the  identification  of  any 
measurement  requirement 
involving  capability  that 
exceeds  the  kown  state  of  the 
art  in  sufficient  time  for  the 
needed  capability  to  be 
developed; 

e)  the  clarification  of 
standards  of  accepatability  for 
all  features  and  requirements, 
including  those  which  contain  a 
subjective  element; 

f)  the  compatibility  of  the 
design,  the  production  process, 
installation,  inspection  and 
test  procedures  and  the 
applicable  documentation; 

g)  the  identification  and 
preparation  of  qualtity  records 
(see  4.16) . 


Relation  to  Other  Contract 
Requirements.  The  inspection 
system  requirements  set  forth  in 
this  specification  shall  be 
satisfied  in  addition  to  all 
detail  requirements  contained  in 
the  statement  of  work  or  in 
other  parts  of  the  contract.  The 
contractor  is  responsible  for 
compliance  with  all  provisions 
of  the  contract  and  for 
furnishing  specified  articles 
which  meet  all  the  requirements 
of  the  contract.  To  the  extent 
of  any  inconsistency  between  the 
contract  schedule  or  its  general 


. 3  Contract  review 

The  supplier  shall  establish  and 
maintain  procedures  for  contract 
review  and  for  the  coordination 
of  these  activities. 

Each  constract  shall  be  reviewed 
by  the  supplier  to  ensure  that 

a)  the  requirements  are 
adequately  defined  and 
documented; 
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o  equivalency 


Document  c 
modifications 

Changes  to  documents  shall  be 
reviewed  and  approved  by  the 
same  functions/organizations 
that  performed  the  original 
review  and  approval  unless 
specifically  designated  other¬ 
wise.  The  designated  organiz¬ 
ations  shall  have  access  to 
pertinent  background  information 
upon  which  to  base  their  review 
and  approval . 

Where  practicable,  the  nature  of 
the  change  shall  be  identified 
in  the  document  or  the 
appropriate  attachments. 

A  master  list  or  equivalent 
document  control  procedure  shall 
be  established  to  identify  the 
current  revision  of  documents  in 
order  to  preclude  the  use  of 
non-applicable  documents. 

Documents  shall  be  reissued 
after  a  practical  number  of 
changes  have  been  made. 


Purchasing 


4.5.1  General 

The  supplier  shall  ensure  that 
purchased  product  conforms  to 
specified  requirements. 

4.5.2  Assessment  of  sub¬ 
contractors 


The  supplier  shall  select  sub¬ 
contractors  on  the  basis  of 
their  ability  to  meet  sub¬ 
contract  requirements,  includ¬ 
ing  quality  requirements.  The 
supplier  shall  establish  and 
maintain  records  of  acceptable 
subcontractors  (see  4.16). 

The  selection  of  sub¬ 
contractors,  and  the  type  and 
extent  of  control  exercised  by 


the  supplier,  shall  be  dependent 
upon  the  type  of  product  and, 
where  appropriate,  on  records  of 
subcontractor's  previously 
demonstrated  capability  and 
performance . 


3.11.2  Purchasing  Documents” 

When,  under  authorization  of  the 
Government  Representative, 
copies  of  the  purchasing 
document  are  to  be  furnished 
directly  by  the  subcontractor  or 
vendor  to  the  Government  Repre¬ 
sentative  at  his  facility  rather 
than  through  Government  chan- 
els,  the  contractor  shall  add  to 
his  purchasing  document  a  state¬ 
ment  substantially  as  follows: 
"On  receipt  of  this  order, 
promptly  furnish  a  copy  to 
the  Government  Represent¬ 
ative  who  normally  serv¬ 
ices  your  plant  or,  if 
none,  to  the  nearest  Army 
Navy,  Air  Force,  or  Def¬ 
ense  Supply  Agency  inspec¬ 
tion  office.  In  the  event 
the  representative  or  of¬ 
fice  cannot  be  located,  our 
purchasing  agent  should  be 
notified  immediately." 


The  supplier  shall  ensure  that 
quality  system  controls  are 
effective . 

4.5.3  Purchasing  data 

Purchasing  documents  shall 
contain  data  clearly  describing 
the  product  ordered,  including, 
where  applicable, 

a)  the  type,  class,  style, 
grade  or  other  precise 
identification; 

b)  the  title  or  other 
positive  identification,  and 
applicable  issue  of  specific¬ 
ations,  drawings,  process  re¬ 
quirements,  inspection 
instructions  and  other  relevant 
technical  data,  including 
requirements  for  approval  or 
qualification  of  product, 
procedures,  process  equipment 
and  personnel; 

c)  the  title,  number  and  issue 
of  the  quality  system 
International  Standard  to  be 
applied  to  the  product. 


3.11.3  Reference  Data.  All 

documents  and  referenced  data 
for  purchases  applying  to  a 
Government  contract  shall  be 
available  for  review  by  the 
Government  Representative  to 
determine  compliance  with  the 
requirements  for  the  control  of 
such  purchases.  Copies  of  pur¬ 
chasing  documents  required  for 
Government  inspection  purposes 
shall  be  furnished  in  accord¬ 
ance  with  the  instructions  of 
the  Government  Representative. 


The  supplier  shall  review  and 
approve  purchasing  documents  for 
adequacy  of  specified 
requirement.:  prior  to  release. 


122 


d)  Functional  testing,  either 
prior  to  or  after  installation, 
or  both,  as  required  by  contract 
to  determine  satisfactory 
operation; 

e)  Identification  and  protect¬ 
ion  from  improper  use  or  dis¬ 
position;  and 

f)  Verifir?  ion  of  quantity. 

7.2.2  Damaged  Government - 
furnished  Material .  The 

contractor  shall  report  to  the 
Government  Representative  any 
Government -furnished  material 
found  damaged,  malfunctioning, 
or  otherwise  unsuitable  for  use. 
In  the  event  of  damage  or  mal- 
fucntioning  during  or  after 
installation,  the  contractor 
shall  determine  and  record 
probable  cause  and  necessity  for 
withholding  material  from  use. 


Product 
traceability 


cation  an 


.4  Process  Controls.  Process 
control  procedures  shall  be  an 
integral  part  of  the  inspection 
system  when  such  inspections  are 
a  part  of  the  specification  or 
the  contract. 


Where  appropriate,  the  supplier 
shall  establish  and  maintain 
procedures  for  identifying  the 
product  from  applicable 
drawings,  specif icaitons  or 
other  documents,  during  all 
stages  of  production,  delivery 
and  installation. 

Where,  and  to  the  extent  that, 
traceability  is  a  specified 
requirement,  individual  product 
or  batches  shall  have  a  unique 
identification.  This  identif¬ 
ication  shall  be  recorded  (see 
4.16). 


Process  contro 
4.8.1  General 

The  supplier  shall  identify  and 
plan  the  production  and,  where 


applicable,  installation 
processes  which  directly  affect 
quality  and  shall  ensure  that 
these  processes  are  carried  out 
under  controlled  conditions. 
Controlled  conditions  shall 
include  the  following: 

a)  documented  work 
instructions  defining  the  manner 
of  production  and  installation, 
where  the  absence  of  such 
instructions  would  adversely 
affect  quality,  use  of  suitable 
production  and  installation 
equipment,  suitable  working 
environment,  compliance  with 
reference  standards/codes  and 
quality  plans; 

b)  monitoring  and  control  of 
suitable  process  and  product 
characteristics  during 
production  and  installation; 

c)  the  approval  of  processes 
and  equipment,  as  appropriate; 

d)  criteria  for  workmanship 
which  shall  be  stipulated,  to 
the  greatest  practicable  extent, 
in  written  standards  by  means  of 
representative  samples. 


4.8.2  Special  processes 

These  are  processes,  the  results 
of  which  cannot  be  fully 
verified  by  subsequent 
inspection  and  testing  of  the 
product  and  where,  for  example, 
processing  deficiencies  may 
become  apparent  only  after  the 
product  is  in  use.  Accordingly, 
continuous  monitoring  and/or 
compliance  with  documented 
procedures  is  required  to  ensure 
that  the  specified  requirements 
are  met.  These  processes  shall 
be  qualified  and  shall  also 
comply  with  requirements  of 
4.9.1. 


control  methods; 

c)  hold  product  until  the 
required  inspection  and  tests 
have  been  completed  or  nec¬ 
essary  reports  have  been  re¬ 
ceived  and  verified  except  when 
product  is  released  under  pos¬ 
itive  recall  procedures  (see 
4.10.1).  Release  under  positive 
recall  procedures  shall  not 
preclude  the  activities  outlined 
in  4.10.2a); 

d)  identify  nonconforming 
roduct . 


4.9.3  Final  inspection  an 
testing 

The  quality  plan  or  documented 
procedures  for  final  inspection 
and  testing  shall  require  that 
all  specified  inspection  and 
tests,  including  those  specif¬ 
ied  either  on  receipt  of  pro¬ 
duct  or  in-process,  have  been 
carried  out  and  that  the  data 
meets  specified  requirements. 

The  supplier  shall  carry  out  all 
final  inspection  and  testing  in 
accordance  with  the  quality  plan 
or  documented  procedures  to 
complete  the  evidence  of 
conformance  of  the  finished 
product  to  the  specified 
requirements . 

No  product  shall  be  despatched 
until  all  the  activities  spec¬ 
ified  in  the  quality  plan  or 
documented  procedures  have  been 
satisfactorily  completed  and  the 
associated  data  and  doc¬ 
umentation  is  available  and 
authorized . 


4.9.4  Inspection  and  test 
records 


3.3  Measuring  and  Test  Equip- 
ment.  The  contractor  shall 
provide  and  maintain  gages  and 
other  measuring  and  testing 
devices  necessary  to  assure  that 
suppliers  conform  to  technical 
requirements.  In  order  to  assure 
continued  accuracy,  these 
devices  shall  be  calibrated  at 
established  intervals  against 
certified  standards  which  have 
kown  valid  relationships  to 
national  standards.  If  product¬ 
ion  tooling,  such  as  jigs, 
fixtures,  templates,  and 
patterns  is  used  as  a  media  of 
inspection,  such  devices  shall 
also  be  proved  for  accuracy  at 
established  intervals. 
Calibration  of  inspection  equip¬ 
ment  shall  be  in  accordance  with 
MIL-C-45662.  When  required,  the 
contractor's  measuring  and  test¬ 
ing  equipment  shall  be  made 
available  for  use  by  the 
Government  Representative  to 
determine  conformance  of  product 
with  contract  requirerents .  In 
addition,  if  condicions  warrant, 
contractor's  personnel  shall  be 
made  available  for  operation  of 
such  devices  and  for  verifi¬ 
cation  of  their  accuracy  and 
condition . 


The  supplier  shall  establish  and 
maintain  records  which  give 
evidence  that  the  product  has 
passed  inspection  and/or  test 
with  defined  acceptance  criteria 
(see  4.16) . 

4. Id  Inspection,  measuring  and 
test  equipment 

The  supplier  shall  control, 
calibrate  and  maintain  inspect¬ 
ion,  measuring  and  test  equip¬ 
ment,  whether  owned  by  the  sup¬ 
plier,  on  loan,  or  provided  by 
the  purchaser,  to  demonstrate 
the  conformance  of  product  to 
the  specified  requirements. 
Equipment  shall  be  used  in  a 
manner  which  ensures  that  meas¬ 
urement  uncertainty  is  known  and 
is  consistent  with  the  required 
measurement  capability. 

The  supplier  shall 

a)  identify  the  measurements 
to  be  made,  the  accuracy 
required  and  select  the  approp¬ 
riate  inspection,  measuring  and 
test  equipment; 

b)  identify,  calibrate  and 
adjust  all  inspection,  meas¬ 
uring  and  test  equipment  and 
devices  that  can  affect  product 
quality  at  prescribed  inter¬ 
vals,  or  prior  to  use,  against 
certified  equipment  having  a 
known  valid  relationship  to 
nationally  recognized  standards 
where  no  such  standards  exist, 
the  basis  used  for  calibration 
shall  be  documented; 

c)  establish,  document  and 
maintain  calibration  proced¬ 
ures,  including  details  of 
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equipment  type,  identification 
number,  location,  frequency  of 
checks,  check  method,  accept¬ 
ance  criteria  and  the  action  to 
be  taken  when  results  are  un¬ 
satisfactory  ; 

d)  ensure  that  the  inspect¬ 
ion,  measuring  and  test  equip¬ 
ment  is  capable  of  the  accuracy 
and  precision  necessary; 

e)  identify  inspection, 
meas-uring  and  test  equipment 
with  a  suitable  indicator  or 
approved  identification  record 
to  show  the  calibration  status; 

f)  maintain  calibration 
records  for  inspection,  meas¬ 
uring  and  test  equipment  (see 
4.16); 

i 

g)  assess  and  document  the 
validity  of  previous  inspection 
and  test  results  when  in¬ 
spection,  measuring  and  test 
equipment  is  found  to  be  out  of 
calibration; 

h)  ensure  that  the  environ¬ 
mental  conditions  are  suitable 
for  the  calibration,  inspect¬ 
ions,  measurements  and  tests 
being  carried  out; 

i)  insure  that  the  handling, 
preservation  and  storage  of 
inspection,  measuring  and  test 
equipment  is  such  that  the 
accuracy  and  fitness  for  use  is 
maintained; 

j)  safeguard  inspection, 
measuring  and  test  facilities, 
including  both  test  hardware  and| 
test  software,  from  adjustments 
which  would  invalidate  the 
calibration  setting. 
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3. *7  Nonconforming  Material.  The 

contractor  shall  establish  and 
maintain  an  effective  and 
positive  system  for  controlling 
nonconforming  material,  includ¬ 
ing  procedures  for  the  identi¬ 
fication,  segregation, 
presentation  and  disposition  of 
reworked  or  repaired  supplies. 
Repair  of  nonconforming  supplies 
shall  be  in  accordance  with 
documented  procedures  acceptable 
to  the  Government .  The 
acceptance  of  nonconforming 
supplies  is  a  prerogative  of  and 
shall  be  as  prescribed  by  the 
Government.  All  nonconforming 
supplies  shall  be  positively 
identified  to  prevent 
unauthorized  use,  shipment  and 
intermingling  with  conforming 
supplies.  Holding  areas, 
mutually  agreeable  to  the 
contractor  and  the  Government 
Representative  shall  be  provided 
by  the  contractor. 


£.12  Control  of  nonconforming 
product 

The  supplier  shall  establish  and 
maintain  procedures  to  ensure 
that  product  that  does  not 
conform  to  specified 
requirements  is  prevented  from 
inadvertent  use  or  installat¬ 
ion.  Control  shall  provide  for 
identification,  documentation, 
evluation,  segregation  (when 
practical),  disposition  of 
nonconforming  product  and  for 
notification  to  the  functions 
concerned . 

4.12.1  Nonconf omity  review  and 
disposition 

The  responsibility  for  review 
and  authority  for  the 
disposition  of  nonconforming 
product  shall  be  defined. 

Nonconforming  product  shall  be 
reviewed  in  accordance  with 
documented  procedures.  It  may  be 

a)  reworked  to  meet  the 
specified  requirements,  or 

b)  be  accepted  with  or  with¬ 
out  repair  by  concession,  or 

c)  regraded  for  alternative 
applications,  or 

d)  rejected  or  scrapped. 

Where  required  by  the  contract, 
the  proposed  use  or  repair  of 
product  (see  4.13.1b)  which  does 
not  conform  to  specified 
requirements  shall  be  reported 
for  concession  to  the  purchaser 
or  his  representative.  The 
description  of  nonconformity 
that  has  been  accepted,  and  of 
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repairs,  shall  be  recorded  to 
denote  the  actual  condition. 

( see  4.16) . 


Repaired  and  reworked  product 
shall  be  reinspected  in 
accordance  with  documented 
procedures . 

'4.13  Corrective  action 

The  supplier  shall  establish, 
document  and  maintain  procedures 
for 

a)  investigating  the  cause  of 
nonconforming  product  and  the 
corrective  action  needed  to 
prevent  recurrence; 

b)  analysing  all  processes, 
work  operations,  concessions, 
quality  records,  service  re¬ 
ports  and  customer  complaints  to 
detect  and  eliminate  potential 
causes  of  nonconforming  product; 

c)  initiating  preventive 
actions  to  deal  with  problems  to 
a  level  corresponding  to  the 
risks  encountered; 

d)  applying  controls  to  ensure 
that  corrective  actions  are 
taken  and  that  they  are  ef¬ 
fective; 

e)  implementing  and  recording 
changes  in  procedures  resulting 
from  corrective  action. 

4.14  Handling,  storage, 
packaging  and  delivery 

4.14.1  General 

The  supplier  shall  establish, 
document  and  maintain  proced¬ 
ures  for  handling,  storage, 
packaging  and  delivey  of 
product . 


4.14.2  Handling 

The  supplier  shall  provide 
ethods  and  means  of  handling 
that  prevent  damage  or  deter¬ 
ioration  . 

4.14.3  Storage 

The  supplier  shall  provide  sec¬ 
ure  storage  areas  or  stock  rooms 
to  prevent  damage  or  det¬ 
erioration  of  product,  pending 
use  or  delivery.  Appropriate 
methods  for  authorizing  receipt 
and  the  despatch  to  and  from 
such  areas  shall  be  stipulated. 
In  order  to  detect  deteriorat¬ 
ion,  the  condition  of  product  in 
stock  shall  be  assessed  at 
appropriate  intervals. 

4.14.4  Packaging 

The  supplier  shall  control 
packing,  preservation  and  mark¬ 
ing  processes  (including  mater¬ 
ials  used)  to  the  extent  nec¬ 
essary  to  ensure  conformance  to 
specified  requirements  and  shall 
identify,  preserve  and  segregate 
all  product  from  the  time  of 
receipt  until  the  supplier's 
responsibility  ceases. 

4.14.5  Delivery 

The  supplier  shall  arrange  for 
the  protection  of  the  quality  of 
product  after  final  inspect-ion 
and  test.  Where  con¬ 
tractually  specified,  this 
protection  shall  be  extended  to 
include  delivery  to  destination. 


.2.2  Records.  The  contractor 
shall  maintain  adequate  records 
of  all  inspections  and  tests. 
The  records  shall  indicate  the 
nature  and  number  of  observ- 


.15  Quality  Records 

The  supplier  shall  establish  and 
maintain  procedures  for  id¬ 
entification,  collection, 


'at ions  made,  the  number  and  type 
jof  deficiencies  found,  the 
quantities  approved  and  rejected 
and  the  nature  of  corrective 
action  taken  as  appropriate. 

3.2.3  Corrective  Action.  The 

contractor  shall  take  prompt 
action  to  correct  assignable 
conditions  which  have  resulted 
or  could  result  in  the 
submission  to  the  Government  of 
supplies  and  services  which  do 
not  conform  to  (1)  the  quality 
assurance  provisions  of  the  item 
specification,  (2)  inspections 
and  tests  required  by  the 
contract,  and  (3)  other 
inspections  and  tests  required 
to  substantiate  product 
conformance . 

3.2.4  Drawings  and  Changes .  The 

contractor's  inspection  system 
shall  provide  for  procedures 
which  will  assure  that  the 
latest  applicalbe  drawings, 
specifications  and  instructions 
required  by  the  contract,  as 
well  as  authorized  changes 
thereto,  are  used  for 
fabrication,  inspection  and 
testing . 


o  equivalency 


indexing,  filing,  storage, 
maintenance  and  disposition  of 
quality  records. 

Quality  records  shall  be  main¬ 
tained  to  demonstrate  achieve¬ 
ment  of  the  required  quality  and 
the  effective  operation  of  the 
quality  system.  Pertinent  sub¬ 
contractor  quality  records  shall 
be  an  element  of  these  data. 

All  quality  records  shall  be 
legible  and  identifiable  to  the 
product  involved.  Quality 
records  shall  be  stored  and 
maintained  in  such  a  way  that 
they  are  readily  retrievable  in 
facilities  that  provide  a  suit¬ 
able  environment  to  minimize 
deterioration  or  damage  and  to 
prevent  loss.  Retention  times  of 
quality  records  shall  be 
established  and  recorded.  Where 
agreed  contractually,  quality 
records  shall  be  made  available 
for  evaluation  by  the  purchaser 
or  his  representative  for  an 
agreed  period. 


4.17  Training 

The  supplier  shall  establish  and 
maintain  procedures  for 
identifying  the  training  needs 
and  provide  for  the  training  of 
all  personnel  activities 
affecting  quality  during 
production  and  installation. 
Personnel  performing  specific 
assigned  tasks  shall  be 
qualified  on  the  basis  of 
appropriate  education,  training 
and/or  experience,  as  required. 
Appropriate  records  of  training 
shall  be  maintained  (see  4.15). 
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4.17  Internal  quality  audits 

The  supplier  shall  carry  out  a 
comprehensive  system  of  planned 
and  documented  internal  quality 
audits  to  verify  whether  quality 
activities  comply  with  planned 
arrangements  and  to  determine 
the  effectiveness  of  the  quality 
system. 

Audits  shall  be  scheduled  on  the 
basis  of  the  status  and 
importance  of  the  activity. 

The  audits  and  follow-up  actions 
shall  be  carried  out  in 
accordance  with  documented 
procedures . 

The  results  of  the  audits  shall 
be  documented  and  brought  to  the 
attention  of  the  personnel 
having  responsibility  in  the 
area  audited.  The  management 
personnel  responsible  for  the 
area  sahll  take  timely 
corrective  action  on  the 
deficiencies  found  by  the  audit 
(see  4.1.3). 


o  equivalency 
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No  equivalency 


3.11.1  Government  Inspection 
Requirements .  When  Government 
inspection  is  required,  the 
contractor  shall  add  to  his  pur¬ 
chasing  document  the  following 
statement : 

"Government  inspection  is 
required  prior  to  ship¬ 
ment  from  your  plant.  Upon 
receipt  of  this  order, 
promptly  notify  the  Gov¬ 
ernment  Representative  who 
normally  services  your  plant 
so  that  appropriate  planning 
for  Government  inspection  can 
be  accomplished." 
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APPENDIX  D 


MIL-Q-9858A/ISO  MATRIX 


MIL-Q-9858 

9001 

9002 

9003 

9004 

1.1,  1.2 

N/A 

N/A 

N/A 

1 

1.3 

4.2 

4.2 

4.2 

5 

2 

N/A 

N/A 

N/A 

N/A 

3.1 

4 .1 .2 .1 

4. 1.2.1 

4. 1.2.1 

4,  5.5,  5.2.3 

3.2 

4.3 

4.3 

N/A 

5.3.3,  8.3,  8.25 

3.3 

4.9.1 

4.8.1 

N/A 

11.3,  17.2 

3.4 

4.16 

4.15 

4.4,  4.6 

5.3.4,  17.3 

3.5 

4.14 

4.13 

N/A 

15 

3.6 

N/A 

N/A 

N/A 

4.3.2,  6 

4.1 

4.5 

4.4 

4.3 

17.2 

4.2 

4.11 

4.10 

4.6 

13 

4.3 

4.11 

4.10 

N/A 

N/A 

4.4 

4.11 

4.10 

N/A 

N/A 

4.5 

4.2 

4.2 

N/A 

N/A 

5.1 

4.6 

4.5 

N/A 

9 

5.2 

4.6.3 

4.5.3 

N/A 

9.5 

6.1 

4.10 

4.9 

N/A 

11.2,  12.1 

6.2 

4.9 

4.8 

N/A 

10,  12.2 

6.3 

4.10.3 

4.9.3 

4.5 

12.3 

6.4 

4.15 

4.14 

4.9 

16 

6.5 

4.13 

4.12 

4.8 

14 

6 . 6 

4.20 

4.18 

4.12 

20 

6.7 

4.12 

4.11 

4.7 

11.7 

7.1 

N/A 

N/A 

N/A 

N/A 

7.2 

4.7 

4.6 

N/A 

N/A 

8  N/A  N/A  N/A  1 
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